Associated Electric Cooperative, Inc. (AECI) Ordered to Pay $42,000 for FAC-008-3 R6 Violation

Instance 1: AECI notified SERC that it was in violation of FAC-008-3 R6. AECI failed to have a Facility Rating that was consistent with its associated Facility Ratings methodology for a single Facility. SERC determined that this noncompliance began under FAC-009-1 R1 and spanned to FAC-008-5 R6.

On April 13, 2021, AECI placed a 161 kV network transmission circuit back in service following the completion of part of a substation rebuild project. The rebuild project consists of several phases of construction, with completion scheduled for July 2022. During weekly meetings to review details of the project, and before the April 13, 2021 energization, AECI personnel contacted the member Generation and Transmission Cooperative (G&T) and verified that the G&T would reuse existing bus work and jumpers for this phase of the project.

On May 5, 2021, after energization, AECI requested confirmation that there had been no changes to any bus conductors and jumpers during this phase of construction. The project did not include jumper modifications for this Facility. Personnel went out in the field and incorrectly verified that the jumper size matched the specification identified by the contractor that provided engineering services for the project. Once G&T personnel received confirmation that the jumpers matched the engineer’s specifications, the G&T responded to AECI that it made no changes to the Facility’s jumpers.

However, on May 27, 2021, during a review of engineering drawings, the G&T identified a possible issue related to the conductor size, which led the G&T to question the jumpers installed by the contractor. The G&T discovered that the third-party contractor hired for construction and inspection of the Facilities did replace a jumper. The contractor deviated from the plans and installed a jumper with a slightly larger physical diameter and lower temperature rating than the existing jumper. The replacement jumper wire installed by the contractor became the most limiting element of the transmission circuit and reduced Facility capacity by 4 percent. AECI did not exceed the correct Facility Rating.

For the extent of condition (EOC) assessment, AECI and the G&T verified all Facility Ratings associated with equipment impacted during the substation rebuild via a field verification. The high temperature conductor is unique to the specific circuit of the noncompliance and is not present at any other location on the AECI transmission system.
The cause of the noncompliance was ineffective project management. The contractor failed to install the jumper that the project documentation specified and AECI failed to review the changes before implementation.

Instances 2-6: On February 14, 2022, AECI submitted finding updates for four additional instances.

Instance 2: On September 27, 2021, during construction, a contractor hired by one of the G&Ts identified a discrepancy between drawings and existing bus work at the same substation described in Instance 1. The legacy drawings correctly identified the 3.5” bus work. The Facility Rating issue occurred due to an error in the grouping of a pair of transformers and a section of bus in the asset management system. The bus work that carries both transformers was incorrectly identified as 5” for the equipment group in the asset management system but was confirmed as 3.5” in the field. The in-service date of the 3.5” bus work was before June 18, 2007. On September 27, 2021, AECI corrected the rating within AECI’s asset management system.

AECI uses ambient adjusted ratings. The incorrect Facility Rating included temperature breakpoints from 68 to 104 degrees Fahrenheit. The incorrect Facility Rating was limited by another piece of equipment that was rated at 836 MVA for those breakpoints. The correct Facility Rating was limited by the 3.5” bus work which ranged from 790 MVA at the 68 degree breakpoint to 649 MVA at the 104 degree breakpoint. The percent derate for the normal rating ranged from 5.5 percent to 22.4 percent at those breakpoints. AECI did not exceed the correct Facility Rating.

For the EOC assessment, AECI and the G&T personnel reviewed other substations with dual transformer groups and did not identify any that converge to a single section of bus work similar to the identified noncompliance.

The cause of the noncompliance was inadequate controls. There was not an internal control in place that would have detected the incorrect entry in the G&T’s asset management system.

Instance 3: On August 23, 2021, during a review of spare equipment for certain transmission Facilities, a G&T identified a switch at a substation for a 161 kV network line that had an inaccurate rating in the G&T’s asset management system. The rating established in the database was 4000 amps, however, the nameplate was 3000 amps. The updated equipment rating derated the line by 2.6 percent. The G&T did not exceed the correct Facility Rating.

For the EOC assessment, G&T personnel inspected the substation in its entirety for equipment rating deviations. Personnel did not identify any additional deviations.

The cause of the noncompliance was a deficient process. Personnel entered the incorrect rating into the Facility Ratings database during the previous FAC-009 mitigation plan. This led to an incorrect equipment rating in the asset management database. AECI’s process lacked steps for verifying equipment rating change impacts before making changes, steps for communicating and coordinating temporary rating changes with System Operations, and quarterly reviews of Facility Rating database requests.

Instance 4: On October 1, 2021, during a review of spare equipment, a G&T identified five inaccurate equipment ratings that impacted overall Transmission Facility Ratings. The five incorrect Facility Ratings resulted in a 21 percent derate and uprates of 5 percent, 18 percent, 30 percent, and 41 percent. The incorrect Facility Ratings date back to before June 18, 2007. AECI did not exceed the correct Facility Rating.

For the EOC assessment, the G&T conducted a field verification of all current transformers (CT) of the G&T’s Bulk Electric System (BES) Transmission Facilities. The G&T’s errors identified during the Transmission resiliency project were limited to CTs. During the previous mitigation plan, the G&T performed walkdowns with substation personnel which may not have had the correct skill set to verify how CTs were wired in the field and thus did not identify these issues during the previous walkdown. The G&T reviewed drawings and equipment manuals to determine the full ratings of the CT’s and the ratio being used. All other equipment was field verified by appropriate substation personnel.

The cause of the noncompliance was inadequate internal controls. G&T personnel entered the data into the G&T’s asset management system from drawings that contained inaccurate information. There were drafting errors and copy and paste issues when creating record drawings from field drawings. When these errors occurred, the G&T hand drew most system protection drawings and handed them over to a drafter to be entered into computer-aided design (CAD) system. The G&T’s post review process was lacking during that time. In 2015, the G&T began to create drawings in CAD and would then send those to the drafters electronically. As part of this new process, the G&T implemented a post drawing review process control to catch these types of errors.

Instance 5: AECI maintains a Modeling and Network Transmission Information System (MANTIS) database of transmission equipment. On August 19, 2019, a transmission planner running the annual TPL dynamics assessment notified the MANTIS team that a G&T owns two circuit breakers at a neighboring entity’s substation. The notification included a recommendation to confirm this information either internally or with the G&T. The G&T verified that it owned two breakers, four switches, and associated equipment at that substation. AECI and the G&T had not modeled this equipment as AECI-owned equipment in MANTIS or the system model.

On November 5, 2020, the MANTIS team requested the G&T to provide the equipment ratings in the asset management system and on December 3, 2020, the MANTIS team confirmed the proper equipment was in the asset management system. On January 13, 2021, the MANTIS team reached out to the neighboring entity to coordinate the required modeling changes to address the ownership issue. On August 29, 2021, the neighboring entity replied to the MANTIS team to coordinate ratings for the modeling changes.

On October 25, 2021, the MANTIS team pushed the ratings changes, which included derates, to the production MANTIS environment. At that time, the G&T discovered that the MANTIS team did not account for the CTs associated with this equipment from the time the G&T provided the data in November 2020, until October 25, 2021.

AECI uses ambient adjusted ratings. The normal rating derate percentage ranged from 40 percent at the lowest temperature breakpoint to 20 percent at the highest temperature breakpoint. AECI did not exceed the correct Facility Rating.

The cause of the noncompliance was an inadequate asset identification process. The process did not include steps that the database team would coordinate and confirm ownership with the G&T and neighboring entity on ties and jointly owned facilities when they are being updated or modified in the database.

For the EOC assessment, all AECI BES ties in which equipment in a substation is owned by more than one entity will be reviewed with the G&Ts and neighboring entities to ensure the proper ownership is documented and a determination of correct Facility Ratings if additional G&T owned equipment is identified.

Instance 6: On July 28, 2022, AECI submitted another finding update. On May 4, 2022, during a quarterly meeting, AECI and the G&Ts determined that G&T personnel were providing PRC-023 relay loadability settings in their asset management systems that the G&Ts calculated at 0.85 per unit (P.U.) which deviates from the 1.0 P.U. voltage identified in AECI’s Facility Ratings Methodology (FRM). Using the 0.85 P.U. relay calculations artificially lowered the equipment ratings for applicable relays in AECI’s Facility Ratings database. The group initiated a project to update all applicable ratings to 1.0 P.U. as designated in the FRM.

The cause of the noncompliance was inadequate internal controls. AECI lacked a control that required applicable personnel to review how relay settings should be evaluated and entered into AECI’s equipment ratings database.

AECI performed a review of relay settings in the Facility Rating database to determine the extent of condition. The incorrect equipment ratings were 17.65 percent below the correct values and impacted 78 Facilities. G&T personnel corrected the Facility Ratings by June 22, 2022. The relay loadability settings were correct and were not modified. The G&T adjusted impacted Facility Ratings to 1.0 P.U. voltage. During the relay settings review, G&T personnel identified 20 additional discrepancies. The discrepancies consisted of five Facility Ratings that were higher than the correct values (ranging from a 7.0-33.9 percent deviation), eleven Facility Ratings that were lower than the correct values (ranging from a 13.0-153.0 percent deviation), three missing relay settings that did not impact the Facility Rating, and one wave trap that had been removed from service by October 2, 2019 but had not been removed from the database.

AECI reviewed three years of historical data for the five Facility Ratings that were rated higher than the correct rating. None of the identified Elements exceeded the correct normal or short-term Facility Ratings during that time.

Lastly, AECI completed the mitigating activity to perform a field verification of current transformers at G&T’s BES transmission facilities and identified 18 additional incorrect equipment ratings that resulted in five incorrect Facility Ratings. Of the five incorrect Facility Ratings, four were derates ranging from 16.37 percent to 19.86 percent, and one uprate of 6.94 percent.

This noncompliance started on June 18, 2007, when FAC-009-1 became mandatory and enforceable, and ended on June 22, 2022, when AECI made the last equipment and Facility Rating correction. The individual Instances had durations of 44 days, 5,215 days, 1,040 days, 5,134 days, 358 days, and 4,393 days, respectively.