Lesson Learned - Breaker Failure Due to Trip Coil Polarity
Below is the full transcript for this episode. If you'd like to review or follow along with the original .pdf version of this NERC Lesson Learned, click here.
EPISODE TRANSCRIPT
Greg Park:From a system operator perspective, it is interesting to understand, or important to understand, that if you don't have a operational trip coil, you cannot open that breaker. You can't open it locally, can't open it remotely, can't open it via relays.
Chris Sakr:Trip coils. They're the heart and soul of circuit breakers. We’ll dive into why in a little bit. But that's pretty important to wrap your head around before going a whole lot further. Without trip coils, breakers are irrelevant pieces of cold steel sitting in a field somewhere getting pooped on by birds and collecting mold. It's bleak, I know, but that's life without trip coils, a cold, dark place. And when they don't function properly, well, you're running the risk of system meltdown, as you'll hear in the following event. On this episode of NERC Lessons Learned, with the help of Greg Park from Northwest Power Pool, we're going to take a close look at “Breaker Failure Due to Trip Coil Polarity,” published March 6th, 2018. Primary interest groups: generator and transmission owners and distribution providers. Before we dive in, I want to make a quick note. This particular lesson learned focuses heavily on a single type of equipment, trip coils, which we'll get into in a minute. It's accompanied by some visual aids: some diagrams, still images, and a graph. All of which can be found on this episode's page on nwpp.org, accompanied by the complete text of the original lesson learned and a transcript of this episode. Okay, so first, before we even get into the problem itself, you need to understand trip coils beyond how sad the world would be without them. Here's Greg with a deeper dive.
Greg Park:What is a trip coil? It's not something you fall over on the ground. It's not a coil of rope. The device that actually initiates a trip, a fast opening of a high voltage circuit breaker is called a trip coil. And what this device does is when the relay senses the fault, it initiates a trip output. And the trip coil is actually what opens the circuit breaker. So, a trip coil is a electromotive of device that is a solenoid. It initiates movement on a latch inside the breaker that, when tripped, will allow that breaker to open up very rapidly. Usually under spring or air pressure. So, that's the first thing to understand when you read this lessons learned of, what does a trip coil do? And it's the actual electromechanical device that will allow the breaker to trip nearly instantaneously when signaled from the relays that there's a problem, or in some cases, if you manually open a breaker, a trip coil is what actually allows that breaker to open up.
Chris Sakr:So, that's all a trip coil is, the device that initiates a breaker trip and opens the circuit breaker. It's not, as Greg said, a coil of rope. Now that we're all on the same page here we can get into the particulars of this lesson learned. We'll start with the problem statement which goes like this: “A temporary phase two phase ground fault occurred on 115 kV line. 115 kV line breaker at one end of the line was slow to operate. Breaker failure protection operated, which caused the remaining 215 kV breakers of a three breaker switching station to open. This left the switching station de-energized. The breaker that failed to operate utilized two trip coils. Onsite investigation indicated that both trip coils had been damaged when they were energized to trip the breaker.” So, that's the problem, which moves us to the details. From here, we'll bounce between my reading of the lesson learned and excerpts from my conversation with Greg where we go a little deeper in clarifying what the heck this actually has to do with you. Spoiler alert, it's complicated. The details begin like this: “The breaker has a common trip coil assembly that was wired, so the polarity was opposing for each trip coil. When the trip coils were energized simultaneously, cancellation of magnetic flux occurred due to this opposingtrip coil polarity, causing the coil armature to remain immobile.” And stop… If you can, this would be the time to check out figures 1A, 1B, and 1C. And while you're deciphering them, what's opposing polarity, magnetic flux, and what about them potentially makes these twisty pieces of metal effect your job behind the desk? Tell us, Greg. Tell us all you know…
Greg Park:What a trip coil is, is the device that pushes or pulls when you apply an electrical current to it. It's a winding that a will activate a plunger basically. But, if you wire these trip coils in opposite, one of them is pushing the trip and one of them is pulling the trip. They go in opposite directions. And the net result is nothing moves in the trip mechanism.
Chris Sakr:So in a perfect world, they're both pushing or they're both pulling?
Greg Park:Yes.
Chris Sakr:Yes. Okay.
Greg Park:It depends on the actual mechanism. They're both operating in the same direction.
Chris Sakr:Okay. So it depends on the end result, but you want both of them to be doing the same thing.
Greg Park:Yes. So in this case, the forces that were exerted by these trip coil solenoids were canceled out and that's why the breaker didn't actually trip. The trip signal was sent correctly. But since the two coils were pushing against each other instead of working together, the breaker failed to trip.
Chris Sakr:So, the trip coils were accidentally wired wrong. Somebody wired them to cancel one another out, and the breaker didn't trip as it was intended. Woof. Moving along with the details, microprocessor relay data indicated that the trip coils were energized by separate primary and secondary relay trip contacts within a time span of less than three milliseconds. You'll see the trip coil assembly in figures three and four. Basically, using the microprocessor data history stored in the relay, they were able to hone in on what went upside down. Greg pointed out a specific detail may have helped in that narrowing down process…
Greg Park:One thing that just dawned on me, three milliseconds was talked about. The trip output was executed very quickly, well within the design Spec of most ... definitely 115 kV breakers. So, that tells you that there wasn't a problem with detection of fault. So, that that piece of information actually adds some information to the story. In that the relay worked correctly, is what that's trying to say.
Chris Sakr:Break that down for me. So, can you explain what exactly that indicates ... how that leads us to this conclusion?
Greg Park:So, when they go out and investigate a misoperation like this, they're going to query that data. And the first thing they want to know is, did the relay adequately detect a fault and send a trip signal to the protective device, which is the trip coils on the breaker? And that's really what that is saying is, in their initial ... in their investigation, it wasn't a relay problem. It was able to detect and send a trip output within three milliseconds.
Chris Sakr:So that's sort of ... it's a process of elimination.
Greg Park:Yes.
Chris Sakr:It's like, "Okay. So, it wasn't this, so now we can move on to thisbeing a possibility."
Greg Park:What that three milliseconds tells the reader is this wasn't a relay problem.
Chris Sakr:Right. And that's an issue that's being dealt with by field personnel? That data analysis is being dealt with by field personnel or who's dealing with that data?
Greg Park:It's technicians and engineers responsible for that work. It may be done locally, may be done via modem. So, if they have dial-in to the relay, they don't even have to leave their desk.
Chris Sakr:Okay.
Greg Park:So I don't like to use the word field personnel.
Chris Sakr:Got It, got it. So what's, like, an operator's line of sight on all of this? To what extent are they involved in sort of trying to diagnose the problem itself? At what point did they enter the equation, besides like, "Oops, something's wrong. I have to react"?
Greg Park:Depends on every entity's process. So every entity, by NERC standards, is really required to evaluate every breaker operation. So, from a system operator perspective, there may be no interaction, there might be a data collection interaction. Since this tripped a breaker failure relay, it was probably pretty well known that there was a potential misoperation at this facility. They might've been asking for operator logs, that kind of thing from an operator during this investigation.
Chris Sakr:So, the real takeaway is, for operators, wouldn't really come until all was said and done, the problem's been corrected, and you're looking back in hindsight.
Greg Park:No. No. So, from an operator perspective, what is interesting is that this led to a breaker failure relay action, which is a very harmful action for a system operator, one you probably see once a year, twice a year in a normal control center.
Chris Sakr:So, through the variables in the equation, the problem itself could be narrowed down further to equipment. The relay worked correctly, which points us even closer to the actual issue. Thing is, this analysis is typically being done by field crews and engineers. An operator wouldn't have a whole lot of visibility on cause until well after the event. So you may ask, does this even have anything to do with me?
Greg Park:A lot of this corrective action is ... really has very little relevance on a system operator sitting at a desk. I think what's interesting about this one is, this is how our equipment operates. And for a system operator, you should know ... have a really good fundamental understanding of how a breaker trips.
Chris Sakr:Yeah. Like basically, just how stuff works.
Greg Park:Yeah.
Chris Sakr:Right.
Greg Park:When you turn the key on a car, what's it actually doing? Right?
Chris Sakr:This is another instance of how something can go wrong and another thing that you can call to mind if something similar is occurring on your system.
Greg Park:You betcha.
Chris Sakr:And now that you hopefully do have a better understanding of how stuff works, let's get into the corrective actions: “The trip coils were replaced and tested to verify proper operation by applying simultaneous trips to them. Simultaneous trip coil energization tests should be conducted on all breakers with multiple trip coils that share a common magnetic path. Care should be observed to simultaneously energize the trip coils for a very short time, the breaker tripping time. One test method option is a test set up, with at least two parallel contacts from an auxiliary relay or switch that could be used and configured to energize two trip coils from their separate DC circuits. Engineering should be consulted for test setup options. And company safety procedures should be followed. If the trip coils are installed properly, the series 52A contacts should interrupt the trip coil currents. If the trip coils are installed improperly, to cancel magnetic flux through the armature, than the test set contacts will be called on to interrupt the trip coil currents. Beware of inductive kickback voltage while interrupting DC current flowing through the trip coils.” So, Greg boiled all this down pretty simply…
Greg Park:What this is really pointing out that when you're doing testing of anything, you can't just test parts of it.
Chris Sakr:Right.
Greg Park:Sometimes you have to test the whole gamut.
Chris Sakr:Right.
Greg Park:It's really important to test the parts. But if it's a functional system, you have to figure out how to test the entire-
Chris Sakr: ... how they interact with each other.
Greg Park:Yeah, so what this missed was, individually, these probably both worked as designed, but together they canceled each other out. For a lot of instances ... This is unique to this one, but it's something you could apply to just about any checkout and test that you would do is, how does this work holistically together when the whole system is being tested?
Chris Sakr:And again, this is happening in the field upon installation, maintenance, things like that. But is there some relevant level of information for an operator to be able to verify that these things have been tested or to be able to point to that upon realizing that there is a problem?
Greg Park:I would say no. The operators are really removed from that process.
Chris Sakr:Right. So, this is all completely just, you know ...
Greg Park:Field personnel.
Chris Sakr:If it goes down, these might be possibilities.
Greg Park:Yeah.
Chris Sakr:Would it be something where like if it went down and the operator might be thinking, "Okay, this might be this." They've called this to memory, they're like, "Oh, maybe it's that and maybe this is the problem." Can they check a log somewhere to see if ...
Greg Park:Generally not.
Chris Sakr:Okay, so they have like zero visibility on this.
Greg Park:The only thing a system operator might be aware of is they're actually doing the testing and the commissioning.
Chris Sakr:So, if you, the operator, are involved in the testing and commissioning of equipment, you can help prevent this in advance by steering toward testing how everything works together as a whole, not just individual parts. Seems kind of self-explanatory, right? Well, apparently not so much. It's probably super clear at this point, the bulk of this lesson learned points directly to the field. There's not a lot a desk operator can really do to curb this. If these important pieces equipment aren't tested as a whole, an operator is stuck dealing with the ramifications in a cold, dark place where alarms are firing off and the world is falling into complete disrepair, getting darker and darker with every second that passes until you're screaming at the screens in front of you. Okay. Maybe not that extreme, but definitely not that fun either, not fun at all. The bulk of the lessons learned are obviously testing related then. Here they are. First, common assembly trip coils installed by a vendor need to be verified. Second, field wiring also needs to be verified, as it may defeat properly installed trip coils. And third, test breakers with common trip coil assemblies for proper tripping when trip coils are energized simultaneously. So yeah, this particular lesson learned, it may feel pretty distant to you and your job at the operator desk, but to close out, Greg's got a little extra insight into where you may be involved and why it's beneficial to keep this on your radar.
Greg Park:For me, what an operator can learn from this is having a questioning attitude about “what is that technician actually testing in the field” and understanding that testing shouldn't just be checking the boxes. We should really be verifying that the equipment's going to work when it needs to work. Is that really something an operator has a lot of control of? Generally, no.
Chris Sakr:But they can have some ... they can at least be able to discerningly beg the question if given the opportunity to.
Greg Park:Yeah. Depends on the culture, of course, of the entity. But you'd really want, hopefully, your operators getting involved in understanding what is actually going on during commissioning of equipment. When you read this, it's amazing to me that this could happen.
Chris Sakr:Why? Talk to that for a second.
Greg Park:Well, it's pretty rudimentary that these key components should be tested to work under that condition of both of them getting a trip output and actually being able to trip the breaker. They didn't do that. If they would have done it, they would have burned up the coils during commissioning.
Chris Sakr:That's it for this episode of NERC Lessons Learned, powered by source.training and produced by Northwest Power Pool. If you're listening on iTunes, SoundCloud or somewhere else, be sure and visit nwpp.org for the full Lesson Learned text and a transcript of this episode. The link's in the show notes. If you like the show, share it with others in the industry who you think might feel the same way. Subscribe to it on your platform of choice and leave us a review and a rating if you have a second. We would love your feedback. Thanks so much for listening and we'll see you next time.