Runaway Fire Hydrant Leaves Power Plant in the Dark
Originally posted May 17, 2014:
Don’t believe it when the Electric Company tells you that the reason your town lost electricity for an hour was because a squirrel climbed onto a transformer and shorted it out. The real reason just may be more bizarre than that and the company doesn’t want you to know all the different creative ways that power can be shut off. This is a tale of just one of those ways. So, get out your pencil and paper and take notes.
Power Plant Notepad
One spring day in 1993 while sitting at the Precipitator computer for Unit one at the Coal-fired Power Plant in North Central Oklahoma, while I was checking the controls to make sure all the cabinets were operating correctly, suddenly there was a distant boom, and the lights in the control room went out. The computer stayed on because it was connected to an electric panel called the VSP or Vital Services Panel, which in turn was supplied by the UPS system (Uninterruptible Power Supply). That was one of those moments where you may pause for a moment to make sure you aren’t still at home dreaming before you fly into a panic.
The Precipitator cabinets all indicated on the computer that they had just shutdown. I rose from the chair and walked around to the front of the Alarm Panel for Unit one, and found that the fluorescent lights were only out on Unit 1. The lights were still on for Unit 2. The Control Panel was lit up like a Christmas Tree with Green, Red, Blue and Yellow Lights. The Alarm Printer was spewing out paper at high speed. As the large sheets of paper were pouring out onto the floor, I watched as Pat Quiring and other brave Power Plant Control Room operators were scurrying back and forth turning switch handles, pushing buttons, and checking pressure gauges.
Just this site alone gave me confidence that everything was going to be all right. These Control Room operators were all well trained for emergencies just like this, and each person knew what their job was. No one was panicking. Everyone was concentrating on the task at hand.
Someone told me that we lost Unit 1, and the Auxiliary Power to Unit 1 at the same time. So, Unit 1 was dead in the water. This meant, no fans, no pumps, no lights, no vending machines, no cold water at the water fountain and most importantly, no hot coffee!!! I could hear steam valves on the T-G floor banging open and the loud sound of steam escaping.
I turned quickly to go to the electric shop to see what I could do there in case I was needed. I bolted out the door and down the six flights of stairs to the Turbine-Generator (T-G) basement. Exiting the stairway, and entering the T-G basement the sound was deafening. I grabbed the earplugs that were dangling around my neck and crammed them into my ears. Steam was pouring out of various pop-off valves. I ducked into the electric shop where across the room Andy Tubbs, one of the electric foreman was pulling large sheets of electric blueprints from the print cabinet and laying them across the work table that doubled as the lunch table.
Andy Tubbs – True Power Plant Electrician
When I asked Andy what happened, I learned that somehow when a crew was flushing out a fire hydrant the water somehow shot up and into the bus work in the Auxiliary Substation (that supplies backup power to the Power Plant) and it shorted out the 189,000 volt substation directly to ground. When that happened it tripped unit 1 and the auxiliary substation at the same time leaving it without power.
An example of part of an Auxiliary Substation
I will explain how a fire hydrant could possibly spray the bus work in a substation in a little while, but first let me tell you what this meant at the moment to not have any power for a Power Plant Boiler and Turbine Generator that has just tripped when it was at full load which was around 515 Megawatts of power at the time.
Normally when a unit trips, the boiler cools down as the large Force Draft (FD) Fans blow air through the boiler while the even larger Induced Draft (ID) fans suck the air from the boiler on the other end and blow the hot air up the smoke stack. This causes the steam in the boiler tubes to condense back into water. Steam valves open on the boiler that allow excessive steam to escape.
When the boiler is running there is a large orange fireball hovering in space in the middle of the boiler. The boiler water is being circulated through the boiler and the Boiler Feed Pump Turbines are pumping steam back and forth between the turbine generator and the boiler reheating the steam until every bit of heat from the boiler that can be safely harnessed is used.
When all this stop suddenly, then it is important that the large fans keep running to cool down the steam, since it is no longer losing energy in the generator as it was when it was busy supplying electricity to 1/2 million people in Oklahoma City. The power is fed to the fans from the Auxiliary substation located right outside the Main Switchgear where all the breakers reside that supply the power to the fans. Unfortunately, in this case, the Auxiliary substation was shutdown as well, leaving the boiler without any fans.
Without fans for cooling, and pumps to circulate the water, the walls of the boiler began heating up to dangerous temperatures. Steam was whistling out of pop off valves, but if the steam drum on the top of the boiler were to run dry, then the entire boiler structure could be compromised and begin melting down. — So, this was serious. Something had to be done right away. It wouldn’t be as bad as the China Syndrome since we were burning coal instead of nuclear power, but it would have caused a lot of damage nonetheless.
From the movie “The China Syndrome” where a similar emergency existed only in the movie, it was a Nuclear Plant
I have a side story about this picture, but I think I’ll save it for another post because I don’t want to digress from the main story at this point (Ok. Let me just say “Jack Maloy and Merl Wright” for those who can’t wait) See the post: “Power Plant Conspiracy Theory“.
With the prospect that the boiler might melt to the ground in a pile of rubble, it would seem that the main priority was to turn the Auxiliary Substation back on so the fans could be turned back on and prevent the boiler from collapsing. So, we walked out to the substation and looked at the switches that would have to be operated in order to first power up the main bus and then to close to supply power to the two big transformers and the six smaller transformers that supplied the Unit 1 Main Switchgear.
While inspecting the switches where the electricity had gone to ground we found that one of the main insulators was cracked.
A High Voltage Insulator like this
Since this insulator was cracked, we didn’t really want to operate the switch to test if another 189,000 volts would go straight to ground again, especially since one of us would be standing right underneath it cranking the switch. So, we went back to the shop to find an alternative.
By this time the Plant Manager, Ron Kilman arrived in the shop, and understanding the urgency to find a solution asked us what were the alternatives. He was relying on our expertise to make the decision.
Ron Kilman is the one on the left in the plaid shirt
The other solution would be to cut the power over from Unit 2 which was still humming away pushing electricity to Oklahoma City out of the 345,000 volt substation. The cut over would be very simple because the switchgear was designed with this in mind. We analyzed the power rating on the auxiliary transformers on Unit 2 and thought that we might be cutting it close to have them running both sets of fans at the same time, especially since the full load amps of a huge fan starting up was about 10 times the normal rate.
The transformer was rated to handle the load, but consider this. What if this caused Unit 2 to trip as well. With the Auxiliary substation offline, if Unit 2 tripped, we would be in twice the amount of trouble we were currently in. What a day it would have been if that had happened and two 250 foot boilers had come crashing to the ground in a pile of rubble. After reading the power ratings on the auxiliary transformers I was thinking, “Yeah, let’s do it! These transformers can handle it.” Andy was not so eager.
So, we were left with one alternative. That was to shut the switch in the Auxiliary substation that had the cracked insulator and take our chances that it wasn’t going to short to ground and blow up over our heads. I think I was eager to close the switch for Andy, but if I remember correctly, he didn’t want me to be the one to suffer the consequences and decided to close the switch himself. Needless to say. Andy closed the switch, and nothing blew up.
High Voltage Switch similar to the one we were closing
As soon as the power was restored to the switchgear, the fans were powered up and the temperature in the boiler was quickly reduced. The coffee pot in the Electric Shop began heating the coffee again. The power plant was saved from a major catastrophe. That was delayed for another day… of which I will talk about later (see the post “Destruction of a Power Plant God).”
So, how exactly does a fire hydrant shoot water up into the bus work of a substation like the picture of the switch directly above? The culprit fire hydrant wasn’t in the substation, it sat alongside it outside the fence a good 50 feet from the high voltage switch. No hose was attached to the fire hydrant. It was only being flushed out as part of a yearly activity to go around and make sure the fire hydrants are all operating correctly.
Here is the story about how the squirrel climbed into the transformer this time….
George Alley, Dale Mitchell and Mickey Postman were going around to the 30,000 fire hydrants on the plant ground (ok. maybe not that many, but we did have a lot of them), and they were opening up the valves and flushing them out. That means, they were letting them run for a while to clear them out from any contaminates that may have built up over the year of not being used.
Throughout their adventure they had opened a multitude of Hydrants situated out in the fields along the long belt conveyor from the coalyard and around the two one-million gallon #2 Diesel tanks.
Large Oil Tank about the size of the two that are at the Power Plant
The brave Power Plant Men, learned that when opening a fire hydrant wide open in the middle of field had unintended consequences. It tended to wash out the ground in front of the flow of the water shooting out of the hydrant. So the team of experts devised a plan to place a board in front of the hydrant when it would be in danger of tearing a hole in the terrain. The board would divert the water into the air where it would fan out and not cause damage to the surrounding area.
This was working fine, and when they arrived at the fire hydrant next to the substation, since the stream from the hydrant was pointing directly into the substation (hmm. a design flaw, I think), they decided to prop the board up against the fence to keep from washing away the gravel in the substation. Well. When a fire hydrant is opened that hasn’t been used for a year, the first flow of water to shoot out is dark brown.
You may think that this is because the water has somehow become dirty over the past year, but that isn’t quite the case. What has happened is that the pipe has been rusting little by little and the water has become saturated with the rust. So, the water shooting out of the hydrant was full of rust (hence the need to flush them out).
Well. Rust is made of metal. Metal is conductive, especially when it is mixed with water. When the water hit the board, it was deflected into the air and happened to direct itself directly into the high voltage switch in the substation. This caused a circuit to the ground which, once it created an arc pumped all the electricity directly into the ground.
Normally when something like this happens it doesn’t trip the Main Power Transformer to a Power Plant.
A Main Power Transformer
This time it did. I know there was a few heads scratching trying to figure it out. I think I figured out what happened a little while later. You see… here is the rest of the story….
Once the unit was back online and the emergency was over, someone finally noticed that the telephone system couldn’t call outside of the plant. Well. I was the main telephone person at the time, so the control room called me and asked me to look into the problem.
I checked the telephone computer and it was up and running just fine. Internal calls could be made. Only any call outside just concluded with a funny humming sound. After checking the circuit in the Logic Room next to the Rolm Telephone Computer I headed for…. guess where….. the Main Switchgear….
A ROLM Phone Computer
In the middle of the main switchgear in the back of the room right next to the Auxiliary Substation beyond the back wall, the outside telephone line came into the plant. The first thing it did was go through a special Telephone Surge Protector.
Telephone Grounding Panel
In this picture above, the silver circular buttons on the left side are really an old style surge protector. whenever there was a power surge, the carbon connection in the surge protector would quickly melt causing the circuit to go straight to ground. Thus protecting the rest of the telephone circuit. So, if some kid in their house decides to connect the 120 volts circuit to the telephone for fun to see what would happen, this circuit would protect the rest of the phone circuits. Keep in mind that this was during the early 1990 when “Surge Protection” still was basically all “mechanical”.
Anyway, when I arrived at this panel and I checked the surge protector to the main line going out of the plant, guess what I found…. Yep. Shorted to ground. Luckily there were some spares that were not wired to anything in the panel and I was able to swap them out for the ones that had been destroyed. — These were a one time use. Which meant, if they ever had to short to ground, they had to be replaced.
Ok. Fine. After a little while, we were able to call back out of the plant, though there was still some residual noise on the line. It was like this… when you called out of the plant, the person on the other end sounded like they were buried in a grave somewhere and they were trying to talk to someone living just like in an episode of the Twilight Episode where a phone line landed on a grave and the dead person tried to call his long lost love from the past.
Gladys Cooper in the Twilight Zone Episode “Night Call”
I didn’t give it much thought other than that I figured the 189,000 volt arc to ground must have shorted out the telephone line since the phone line ran directly under the auxiliary substation ground grid.
It wasn’t until the next morning when the Southwestern Bell repairman showed up at the plant. I knew him well, since he had been working on our phone lines since before the AT&T breakup in 1984. When I met him in the front of the electric shop, he said that he needed to check our telephone circuits. I told him that I knew that we had a problem because we had a high voltage short to ground yesterday and I found our surge protectors melted away.
He explained to me that not only was our circuit affected, but that every relay house from here to Ponca City was blown out. That’s when I realized that the problem was the reverse of the usual situation. What had happened was that the Ground Grid in the substation and the surrounding area (including the Unit 1 Main Power Transformer) had become hot. What do you do when the ground grid becomes charged?
The Ground Grid is what is supposed to protect you when a surge happens, but what happens when the ground grid itself is the problem? In this case, when the high voltage line about 60 feet from the telephone cable surge protector, arced to ground, it fed a tremendous amount of power back through the ground grid. when equipment detected the surge in voltage, they automatically defaulted their circuits to ground. That’s why the telephone circuit died. That’s what tripped the Main Power Transformer.
When the telephone circuit detected the high voltage surge, it shorted to ground (which was the problem), causing the high voltage to feed directly into the phone line and down the line to the next Southwestern Bell relay switch, which also defaulted to ground, trying to bleed off the surge as it went from relay switch to switch until enough of the power was able to be diverted to ground.
That day sure turned out to be a learning experience. I learned that when all the lights go out in the control room, that it is almost assured that the coffee pot in the electric shop is going to stop working. I also learned that in order to coax the plant manager to the electric shop, a major electrical tragedy is one good way. I learned that when shooting rusty water into the air don’t point it at a high voltage auxiliary substation switch. — I’m sure Mickey Postman learned that lesson too. I also learned that just like in Star Trek… whenever there is a dangerous job to do, the Captain is always the one that wants to do it. Does that make sense? Send a Peon like me in there…
I also learned something else about Power Plant Men…. You see…. People like Dale Mitchell, George Alley and Mickey Postman all are examples of incredibly wonderful Power Plant Men. When they were out there doing their duty and something tragic like this, all the Power Plant Men felt their pain. They knew that they all felt guilty for tripping the unit. It didn’t matter that a million dollars every so many minutes was walking out the door in revenue. The only thing that mattered was that these three men were safe.
Mickey Postman
Since I have left the Power Plant, I have found that the idea that the employee is the greatest asset that a company can possess is not a universal idea. You see, there was never the thought that any of these people should be fired for their mistake. On the contrary. The true Power Plant Men did whatever they could to let them know that they knew exactly how they felt. It could have happened to any of them.
Besides the friendship between Power Plant Men, one of the things I miss most about working at the Power Plant is that the employees are held in high esteem as a real asset to the company. Many could learn from their example.
Comments from the Original post
-
-
I like the mix of storytelling and information sharing you deliver here. Thanks again
-
-
-
Aug 8, 2011 I lost all reserve in and lost both units in 2 separate storms and A1 would not start for a few minutes and A2 was leaking antifreeze terribly not a scratch and back online in 34 hours both units and no one ever asked me one question even with a Safety Dept not one question asked of me the SS on duty. Not many men on the planet have ever experienced an uncontrolled total plant outage- you would of thought a learning opportunity would of took place. Feb 2011 worst winter temps in years 50 Units or more tripped in Texas let alone the trusty units of Redbud and McClain were fighting Sooner rolled right along with storm warnings for two weeks ahead – the ICS still went to Detroit for just tours of other facilities once again the fall guy Tarver McArthur stood alone. I had authored a Freeze Protection Plan for the plant and that seemed to save the day and explain to the regulatory bodies how we were online and everyone else in Texas was off and enjoying rolling blackouts in a terrible winter weather situation not to mention our powers that be were all stranded in Detroit and very few people could get to the plant without getting stuck trying to get there as well – but a few health heart issues later I am still here to tell about it all you would think folks would want to take advantage of someone that had went through the fire and Ice but thats ok I want them someday distant to get all the credit they deserve when the trumpets sound as that is truly what matters most. DT
-
-
Great story, and yes, you took exactly the proper lesson from it, and it is too bad that many of our bosses haven’t learned it.
And yup, I’ve buried a lot of electrocuted squirrels over the years.
Psychological Profile of a Power Plant Control Room Operator
Originally posted on July 6, 2012:
I suppose that many parents while raising their children would hear them say, “Dad, can you read that story to us again about the pirates that go to the island to find the treasure but Jim Hawkins fights them single-handed?” Or their children might say to their mother, “Will you tell us the story again about how you met daddy?” In my household, my children would say, “Dad, tell us another story about how you played a joke on Gene Day when you were working at the Coal-Fired Power Plant in Oklahoma? Those are always the best!”
As I pointed out in my third post this year called “Power Plant Humor And Joking With Gene Day” the first time I met Gene Day, I could tell that he was the type of person that would take a joke well, or so I thought…. One of the favorite stories my daughter would like for me to tell her as she was growing up was the one where I had created a Psychological Profile of Gene Day, who at that time was an Auxiliary Operator.
It began one day when I was leaving the electric shop through the Turbine Generator (T-G) building ground floor. A very noisy location as large steam pipes wound around under the Turbines where the steam caused a rumbling or whining sound. It was normal when walking through this area to reach up to the ear plugs that were draped over your shoulders and put them in your ears because the decibels were dangerously high if you were exposed earplugless too long.
Earplugs on a cord that can be draped over your shoulders when not in use.
I stepped from the landing leading from the electric shop and started toward number 1 boiler when I spied Gene Day making his way around the first floor of the T-G building inspecting equipment and marking his documents indicating that they were operating correctly. As I saw him turn toward my direction I quickly dodged behind the nearest metal pillar (I-Beam). I peaked my head out from behind the pillar and took out the notepad that was in my back pocket, and the pen from my vest pocket pocket protector.
A pocket protector is a must for electricians and computer nerds who need a place to keep their small tools.
Gene saw me and gave me a suspicious look as I began feverishly writing in my notepad while looking from my notepad back to Gene and then back to what I was writing. After I had done this for about 10 seconds or so, I put my pad away, my pen back in the pocket protector and strolled away toward unit 1 to continue my work.
A notepad like this
It happened that this particular week was Gene’s week to monitor the equipment in the T-G building, so throughout the week he would be making his way somewhere around the T-G building with his clipboard in hand. Each time I encountered him I would do the same thing. I would visibly hide behind a beam and write notes in my notepad. I saw Gene rather frequently during the week because the majority of the time, I left the electric shop by going through the T-G room to one of the boilers and then to the precipitators, where I spent most of my time working at this time in my career (but that is another story for a later time).
Each time, Gene would watch me suspiciously knowing that I was just messing with him, but not exactly sure what I was “up to”. At the time, I wasn’t sure either. So I just wrote down what I saw Gene doing, that way, if he ran over and grabbed my notepad from me, it wouldn’t say anything other than what I saw.
It had happened at the plant a few years earlier when I was a janitor that the company had hired an efficiency expert to monitor the employees at the plant. He would walk around the plant with a stopwatch observing the employees. When he saw them he would write notes on his clipboard. It became very unnerving because you would walk around the corner and there he would be standing writing something down about you.
I went to the Assistant Plant Manager Bill Moler and told him that this creepy guy keeps showing up in the main switchgear by the janitors closet. And every time he sees me, he writes something down. I told Bill that it really bothered me. He explained that he is an efficiency expert and he has a certain path that he takes throughout the day and takes a snapshot of what the workers are doing at that moment and writes it down. By doing that he calculates how efficient we are. It seemed pretty silly to me, because most mechanics when they saw him coming put their tools down and did nothing while he walked by until he was out of sight again.
When I was on the labor crew a few weeks later, and I was blowing coal dust off of all the I-Beams above the bowl mills with a high pressure air hose, I looked down, and through my fogged up goggles I could see this guy standing directly under me. I was about 50 feet above him crawling across an I-Beam with the air hose blowing black dust everywhere. He had crossed my barrier tape to go into the bowl mill area to see how efficient I was being.
This is the type of barrier tape I was using. It is made of woven plastic fibers.
I was so mad I turned off my air hose, climbed down the wall Spiderman-like (no. not head first) and went straight into the A-Foreman’s office and told Marlin McDaniel that the Efficiency Expert had crossed my barrier tape and was standing directly under me as I was blowing down the beams with air.
It turned out that the Efficiency Expert had gone upstairs to complain that some guy in the bowl mill had dumped a bunch of coal dust on him while he was monitoring him from below. Evidently, he wasn’t expert enough to know you weren’t supposed to cross someone’s barrier tape without permission, as was indicated on the caution tags that were tied to the barrier tape. From that point on, the efficiency expert (now in lowercase) had to be accompanied by someone from the plant to make sure he wasn’t breaking any safety rules and putting himself at harm.
To make a long side story short, we turned out to be so efficient, people came from all over the world to study us. Somebody downtown hired the efficiency expert full time, but later he was laid off during the first downsizing.
He reminds me of a person on an episode of Star Trek The Next Generation where this guy Lieutenant Commander Remmick comes aboard the Enterprise and he walks around inspecting everything and asking everyone questions that make them uncomfortable and at the end asks Picard if he could come work on the Enterprise. He looked so much like him, that I thought maybe our efficiency expert went to Hollywood to become an actor.
Lieutenant Commander Remmick
Back to Gene Day. I suppose the thought of the efficiency expert may have been going through my mind as I was taking notes about Gene Day at work. Like I said, at the time, I didn’t know what I was going to do with it.
It finally came to me on Thursday morning. This was the last day that Gene was going to be on the day shift, so I figured I might as well do something about it. So when I entered the shop that morning, I sat down at the desk of my foreman Andy Tubbs and began to write. The title at the top of the page was: “The Psychological Profile of Gene Day”.
Using the notes I had taken during the week, I wrote things like the following: “Gene Day walks around the Turbine Generator building with a clipboard in his hand trying desperately to look like he’s doing something important. He constantly hopes that someone is watching him because he dislikes doing so much work to act busy for no reason.
At times Gene Day gets paranoid and believes that he sees people spying on him from behind every corner (especially I-Beams). Sometimes Gene Day stands in the middle of the T-G floor staring up into space as if he forgot what he was supposed to be doing.” I’m sure I wrote more, but I don’t remember everything… but I do remember the second from the last sentence. I will save that for a couple of paragraphs from now.
I took my Psychological Profile of Gene Day and went up to the Control Room. I noticed out of the corner of my eye that Gene had just walked over toward the break room behind the Auxiliary Control Panel so I walked over by the Shift Supervisor’s office and I leaned against the top of the large Blue Monitor and placed the Psychological Profile on the top of the monitor in front of me.
Less than a minute later Gene came walking around behind me and seeing the paper on the top of the monitor came up behind me and looked over my shoulder and began to read….. He wasn’t trying to hide the fact that he was reading the paper, and I obviously knew he was there, and the title in large bold letters did say, “The Psychological Profile of Gene Day”. So, he read on.
I heard a few chuckles as he read through my interpretation of what he had done during the week. Then he came to the “Second from the Last Sentence”, as I could hear him reading quietly in my ear… The sentence read, “Gene Day sneaks up behind people and reads their private material over their shoulder!” — Bingo! I had him!
When he read that he grabbed me by the throat and started to Throttle me! Shaking me back and forth. This would have been a humdinger of a joke at that point, but I had one more sentence up my sleeve… ur… I mean on the paper….
As I was waivering (is that a word?) back and forth between life and death I managed to eke out something like: “Wait! There’s More!”…. Gene Day let up on me a little and looked down at the page and read the last sentence…… It read….. “Gene Day tries to strangle people who are only trying to help him by creating his Psychological Profile.”
That was all it took. Another perfect joke played on Gene Day, and I was able to live to tell about it. When Gene read that he was stunned into dismay. Giggling as hard as he could he retreated shaking his head in defeat.
Now, I know that Gene reads these posts, and he may remember this story a little differently, and I’ll give him that because Gene is older than dirt and his memory isn’t that good. But I was the one that was being strangled, and I still have a vivid image of those few moments. Not only do I, but so do my children, who will one day tell their children, who will say, “Mommy, will you tell us that story again about how Grandpa was strangled by Gene Day that time in the Control Room at the Coal-Fired Power Plant in Oklahoma?”
Resistance in a Coal-Fired Power Plant
Originally posted April 19, 2013:
Resistance is Futile! You may have heard that before. Especially if you are a Star Trek Fan. If not, then you know that there is always some form of resistance wherever you are.
Captain Picard as Locutus trying to convince you that “Resistance is Futile” Like that is ever going to happen
I learned a lot about resistance when I first joined the electric shop at the coal-fired power plant in North Central Oklahoma in 1984. I was assigned to work with Sonny Kendrick and Bill Rivers on the Precipitator during overhauls and when I wasn’t working on the manhole pumps and there wasn’t any other emergencies going on. Actually, from 1984 on, on the Precipitator for the next 17 years I continued to work on the precipitator… (if I had only known my fate….).
Not only did I learn a lot about resistance, I also learned about capacitance, reactance, transformers, rectifiers, power supplies, diodes, transistors, op amps, and pots (also known as potentiometers). Bill Rivers was the brains of the outfit. Sonny was the Electric Specialist banished to the Precipitator by Leroy Godfrey (See Singing Along with Sonny Kendrick). Bill thought up the ideas and Sonny went to work to implement them. I just jumped in where I was needed.
The Precipitator is the large box between the boiler and the smokestack (maybe you can see this in the Power plant picture). The purpose of the electrostatic precipitator is to take the smoke (or fly ash) out of the exhaust before it went out of the smokestack.
The plant has a similar electrostatic precipitator, only ours is twice as long
The controls for the Precipitator were all electronic at that time. That meant that there were circuit boards full of resistors, capacitors, transistors, operational amplifiers, diodes and potentiometers. These circuit boards controlled the way the power was distributed throughout the precipitator wires and plates through high powered transformers, and how the rappers and vibrators operated that dropped the collected ash into the hoppers.
Bill had me take an electronics course at the Indian Meridian Vo-tech so I would know the basics. Then he taught me all the shortcuts. I had to be able to look at a resistor and tell right away what the value of resistance it was. Resistors are color-coded and you had to learn what each of the colors represented…
This for instance is a 339 ohm resistor with a 5% tolerance.
I was expected to know this by sight. Bill would test me. There was a mnemonic device that I was taught to remember what each color represented, but it is not appropriate to repeat it, so I won’t. It is enough to say that the colors go like this: Black, Brown, Red, Orange, Yellow, Green, Blue, Violet, Gray, White (I will never forget this my entire life). These represent the numbers zero through 9. Here is a full explanation of how to read a resistor….. just in case you are curious, or you are such a boring person that you really need some material to bring up when you are at a party and don’t know what to say:
This is how the color codes work for resistors works
I found that having just the correct amount of resistance was very important. Too much or too less, and everything stops working.
Isn’t it that way with management also? If the management is too resistant to change, then things come to a halt. If they have too little resistance, they lose control of the situation. Depending on the circuit (or managerial decision) and what you are trying to do, it helps to have a manager that has a variable resistance to meet the needs of each situation. Resistance to change is always a balancing act.
During the first two years I was an electrician, the main control panels that controlled the operation of the precipitators were electronic. We spent a lot of time in the lab troubleshooting electric circuits looking for blown (or bad) parts that needed to be replaced. Then we would solder new components on the circuit boards and then put them back in operation. I learned how to be an electronics junky. I became addicted to fixing electronic circuit boards. It was like a game to me.
Later, the precipitator controls were changed to digital controls. That is, they were more like little computers controlling the precipitator. Instead of a bunch of circuit boards dumbly, but cleverly, doing their job, (how many commas can I use in one sentence?), little brains were added that made decisions and reacted to conditions in a much more dynamic way.
What was interesting was that one day Bill Rivers was describing how technology was going to be in the future. He said that some day, we will be able to sit in the lab and look on a computer and see what all the controls in the precipitator were doing (this was 1984). If something isn’t working right, we could just reach over, type a few keys on the computer and adjust the controls. Drink our sweet tea (a necessary staple in Oklahoma at the time), and then wait for the next crisis…. Then he would giggle at the look of disbelief on my face.
When he was telling me this, I was thinking in my head…. Well, that would be nice, but this sounds more like a pipe dream to me than reality. What does an older guy with six kids from a tool and die company in Columbia Missouri (where I grew up, by chance) know about the future of anything….. well…. anything…uh… new age…. If that is what you might call it… I found out you just don’t really know when you are sitting in front of a true “visionary” with tremendous insight.
Bill Rivers had this incredible knack for telling the future. In 1984 he was predicting computer controls in the control room where you ran the entire plant from a computer on a desk instead of using the “Big Board”.
I love this picture! It makes me feel at home! This was not our plant, but is a Power Plant control room
He said you would be able to call someone on a phone you kept in your pocket or your watch like Dick Tracey.
Dick Tracy in a control room of a power plant sending commands to the hopper nozzle booster pumps
I don’t know what journals Bill was reading or if he just dreamed all this stuff up in his head, or maybe he was a Star Trek Fan that believed that if you can dream it up you can do it. I do know that he picked up on subtle queues and made great inferences from them that seemed astronomically unlikely. However, I have to admit that he caught me off guard a number of times with predictions that definitely came true.
I will talk about this more in a future post, but for now I will say that we did upgrade the precipitator to where you could sit in the control room and monitor and adjust the precipitator controls (all 84 on each unit), and even each of the rappers (672 rappers) and vibrators (168 vibrators) on the roof of each precipitator. With one key on the computer I could send a plume of ash out of the smokestack that looked like the unit had just tripped, and a moment later, clean it up again. This meant that I could send smoke signals to the Osage Indian tribe 20 miles north up the Arkansas (pronounced “Are Kansas”) river, telling them that the Pow Wow would begin at sunset.
Today, I understand that the “Big Board” at the plant is just a large junction box and the plant is controlled almost (if not) completely by computers sitting on the desk. Before I left the plant in 2001, this was being transitioned slowly to computer controls. I have another story to tell some day about this, and how an operator named Jim Cave, a Power Plant Genius and true Power Plant Man of the highest integrity, was snubbed by upper management for speeding this technology along. — Another example of Power Plant Resistance….
But for now…. back to my electronic days… before I began re-programming the Eeprom chips in the precipitator controls….
An Eeprom Chip used in the precipitator controls which I had a very “personal” relationship with…
Bill Rivers confided with me one day that when the new Instrument and Controls department had been formed from the “Results” department that his dream had been to become a part of this team. It meant the world to him. It was where he believed he belonged. It was one of his major goals in life.
There used to be two electrical specialists in the Power Plant. Sonny Kendrick was not always the only one. The other specialist was chosen to go to the Instrument and Controls shop. Bill Rivers wanted to move there also. He definitely had the experience and the knowledge to be a superb instrument and controls person. But Bill had this one problem.
He loved to joke around. He loved to pull strings and push buttons. I have mentioned in a previous posts that Bill would play a new joke on Sonny Kendrick every single day. As I have unfortunately found out in my own life… this tends to make them…. well….. it tends to make enemies out of those who have a chip on their shoulder. Those people who naturally feel inadequate in their abilities or their position in life. To go one step further…. anyone who feels “unloved”….. these people definitely do not like being joked with. They seem to never forgive you. My greatest regret in life is joking around with these individuals.
So, when it came time to choose who would be a part of the new Instrument and Controls shop, Bill Rivers was turned down. It was explained to him that the reason he was not given the job was because he cut off the leads of a transistor when he replaced them. — I’m not kidding. Bill Rivers had the habit of cutting off the leads of each resistor, transistor, diode or capacitor that he replaced…. this is why Monty Adams turned down his request for joining the “elite” Instrument and Controls shop (as he told Bill to his face).
Someone had told the Instrument and Controls Supervisor Monty Adams that Bill Rivers cut the leads off of transistors and resistors when he replaced them so that you couldn’t test them to see if they were all right. Implying that he didn’t want you to know whether he had replaced the transistor or resistor by mistake.
Bill Rivers took several transistors, cut the leads off of each of them and handed them to me and asked me to test them to see if they were still good or if they were bad. I took out my voltmeter, set it to ohms, and proceeded to test them as Bill Rivers had taught me. I told him…. this transistor is good….. this one is bad….
You see…. there is no way to cut the leads off of a transistor in such a way to make it impossible to tell if a transistor is good or bad…. In reality…. you cut the leads off of a bad transistor so that the person working on the circuit board knows that this is a bad transistor and doesn’t use it again by accident. This was electronics 101.
When Bill told me this story, he literally had tears in his eyes. This was because being part of the Instrument and Controls team was part of his dream. His family and the entire rest of his life was decided the day he was told that he was not going to be a part of a team that he believed was his true lot in life.
I remember his exact words as he sat there in the lab alone and told me this story. He said, “… and Monty didn’t know… He didn’t know that you cut the leads…. that is standard procedure….” In Bill’s giggly way, he was crying out loud as he told me this.
From that point on….I knew that the decisions Bill made in his life were driven by that one decision to exclude him from this team. Unlike many of us that could say to ourselves…. “That is their loss”…. Bill kept this pain in his heart each day…. Every decision from that day further was effected by this event.
I calculated it out one day that I spent 414 hours driving back and forth from Stillwater, Oklahoma to the plant and back each day with Bill Rivers (along with Yvonne Taylor and Rich Litzer and occasionally others that needed a ride), and over that time, I became very close to Bill, even to the point of tutoring his son in Algebra (see post: How Many Power Plant Men Can You Put in a 1982 Honda Civic?).
I say this because I know about the pain that inflicted Bill River by a rash decision based on the hearsay of someone that held a grudge. I know how his entire life was changed and how it ended for Bill Rivers as a power plant employee. I know that every decision by Bill after this date was made in response to this one decision. Anyone who experienced Bill after 1983 knows what I am talking about.
I realized that today my own decisions in life help spell out my future. How some little remark may be misinterpreted, or even properly so. I realize as I write this post that how I accept or reject these events in my life, determines the future of my family. After seeing how every event in Bill’s life after that day at the power company was determined by his experience was to his detriment, I am determined not to let the same thing happened to me…..
That is why I have taken on the philosophy in my life that no matter how my actions are misinterpreted, I am determined to remain true to myself. I know what I mean, and I mean what I say, and I say what I mean, and an Elephant is Faithful 100 %.
From Horton Hatches an Egg by Dr. Seuss
Comment from the Original Post:
-
It’s amazing how many decisions are made based on incorrect / incomplete information (at all levels).
Runaway Fire Hydrant Leaves Power Plant in the Dark
Originally posted May 17, 2014:
Don’t believe it when the Electric Company tells you that the reason your town lost electricity for an hour was because a squirrel climbed onto a transformer and shorted it out. The real reason just may be more bizarre than that and the company doesn’t want you to know all the different creative ways that power can be shut off. This is a tale of just one of those ways. So, get out your pencil and paper and take notes.
Power Plant Notepad
One spring day in 1993 while sitting at the Precipitator computer for Unit one at the Coal-fired Power Plant in North Central Oklahoma, while I was checking the controls to make sure all the cabinets were operating correctly, suddenly there was a distant boom, and the lights in the control room went out. The computer stayed on because it was connected to an electric panel called the VSP or Vital Services Panel, which in turn was supplied by the UPS system (Uninterruptible Power Supply). That was one of those moments where you may pause for a moment to make sure you aren’t still at home dreaming before you fly into a panic.
The Precipitator cabinets all indicated on the computer that they had just shutdown. I rose from the chair and walked around to the front of the Alarm Panel for Unit one, and found that the fluorescent lights were only out on Unit 1. The lights were still on for Unit 2. The Control Panel was lit up like a Christmas Tree with Green, Red, Blue and Yellow Lights. The Alarm Printer was spewing out paper at high speed. As the large sheets of paper were pouring out onto the floor, I watched as Pat Quiring and other brave Power Plant Control Room operators were scurrying back and forth turning switch handles, pushing buttons, and checking pressure gauges.
Just this site alone gave me confidence that everything was going to be all right. These Control Room operators were all well trained for emergencies just like this, and each person knew what their job was. No one was panicking. Everyone was concentrating on the task at hand.
Someone told me that we lost Unit 1, and the Auxiliary Power to Unit 1 at the same time. So, Unit 1 was dead in the water. This meant, no fans, no pumps, no lights, no vending machines, no cold water at the water fountain and most importantly, no hot coffee!!! I could hear steam valves on the T-G floor banging open and the loud sound of steam escaping.
I turned quickly to go to the electric shop to see what I could do there in case I was needed. I bolted out the door and down the six flights of stairs to the Turbine-Generator (T-G) basement. Exiting the stairway, and entering the T-G basement the sound was deafening. I grabbed the earplugs that were dangling around my neck and crammed them into my ears. Steam was pouring out of various pop-off valves. I ducked into the electric shop where across the room Andy Tubbs, one of the electric foreman was pulling large sheets of electric blueprints from the print cabinet and laying them across the work table that doubled as the lunch table.
Andy Tubbs – True Power Plant Electrician
When I asked Andy what happened, I learned that somehow when a crew was flushing out a fire hydrant the water somehow shot up and into the bus work in the Auxiliary Substation (that supplies backup power to the Power Plant) and it shorted out the 189,000 volt substation directly to ground. When that happened it tripped unit 1 and the auxiliary substation at the same time leaving it without power.
An example of part of an Auxiliary Substation
I will explain how a fire hydrant could possibly spray the bus work in a substation in a little while, but first let me tell you what this meant at the moment to not have any power for a Power Plant Boiler and Turbine Generator that has just tripped when it was at full load which was around 515 Megawatts of power at the time.
Normally when a unit trips, the boiler cools down as the large Force Draft (FD) Fans blow air through the boiler while the even larger Induced Draft (ID) fans suck the air from the boiler on the other end and blow the hot air up the smoke stack. This causes the steam in the boiler tubes to condense back into water. Steam valves open on the boiler that allow excessive steam to escape.
When the boiler is running there is a large orange fireball hovering in space in the middle of the boiler. The boiler water is being circulated through the boiler and the Boiler Feed Pump Turbines are pumping steam back and forth between the turbine generator and the boiler reheating the steam until every bit of heat from the boiler that can be safely harnessed is used.
When all this stop suddenly, then it is important that the large fans keep running to cool down the steam, since it is no longer losing energy in the generator as it was when it was busy supplying electricity to 1/2 million people in Oklahoma City. The power is fed to the fans from the Auxiliary substation located right outside the Main Switchgear where all the breakers reside that supply the power to the fans. Unfortunately, in this case, the Auxiliary substation was shutdown as well, leaving the boiler without any fans.
Without fans for cooling, and pumps to circulate the water, the walls of the boiler began heating up to dangerous temperatures. Steam was whistling out of pop off valves, but if the steam drum on the top of the boiler were to run dry, then the entire boiler structure could be compromised and begin melting down. — So, this was serious. Something had to be done right away. It wouldn’t be as bad as the China Syndrome since we were burning coal instead of nuclear power, but it would have caused a lot of damage nonetheless.
From the movie “The China Syndrome” where a similar emergency existed only in the movie, it was a Nuclear Plant
I have a side story about this picture, but I think I’ll save it for another post because I don’t want to digress from the main story at this point (Ok. Let me just say “Jack Maloy and Merl Wright” for those who can’t wait) See the post: “Power Plant Conspiracy Theory“.
With the prospect that the boiler might melt to the ground in a pile of rubble, it would seem that the main priority was to turn the Auxiliary Substation back on so the fans could be turned back on and prevent the boiler from collapsing. So, we walked out to the substation and looked at the switches that would have to be operated in order to first power up the main bus and then to close to supply power to the two big transformers and the six smaller transformers that supplied the Unit 1 Main Switchgear.
While inspecting the switches where the electricity had gone to ground we found that one of the main insulators was cracked.
A High Voltage Insulator like this
Since this insulator was cracked, we didn’t really want to operate the switch to test if another 189,000 volts would go straight to ground again, especially since one of us would be standing right underneath it cranking the switch. So, we went back to the shop to find an alternative.
By this time the Plant Manager, Ron Kilman arrived in the shop, and understanding the urgency to find a solution asked us what were the alternatives. He was relying on our expertise to make the decision.
Ron Kilman is the one on the left in the plaid shirt
The other solution would be to cut the power over from Unit 2 which was still humming away pushing electricity to Oklahoma City out of the 345,000 volt substation. The cut over would be very simple because the switchgear was designed with this in mind. We analyzed the power rating on the auxiliary transformers on Unit 2 and thought that we might be cutting it close to have them running both sets of fans at the same time, especially since the full load amps of a huge fan starting up was about 10 times the normal rate.
The transformer was rated to handle the load, but consider this. What if this caused Unit 2 to trip as well. With the Auxiliary substation offline, if Unit 2 tripped, we would be in twice the amount of trouble we were currently in. What a day it would have been if that had happened and two 250 foot boilers had come crashing to the ground in a pile of rubble. After reading the power ratings on the auxiliary transformers I was thinking, “Yeah, let’s do it! These transformers can handle it.” Andy was not so eager.
So, we were left with one alternative. That was to shut the switch in the Auxiliary substation that had the cracked insulator and take our chances that it wasn’t going to short to ground and blow up over our heads. I think I was eager to close the switch for Andy, but if I remember correctly, he didn’t want me to be the one to suffer the consequences and decided to close the switch himself. Needless to say. Andy closed the switch, and nothing blew up.
High Voltage Switch similar to the one we were closing
As soon as the power was restored to the switchgear, the fans were powered up and the temperature in the boiler was quickly reduced. The coffee pot in the Electric Shop began heating the coffee again. The power plant was saved from a major catastrophe. That was delayed for another day… of which I will talk about later (see the post “Destruction of a Power Plant God).”
So, how exactly does a fire hydrant shoot water up into the bus work of a substation like the picture of the switch directly above? The culprit fire hydrant wasn’t in the substation, it sat alongside it outside the fence a good 50 feet from the high voltage switch. No hose was attached to the fire hydrant. It was only being flushed out as part of a yearly activity to go around and make sure the fire hydrants are all operating correctly.
Here is the story about how the squirrel climbed into the transformer this time….
George Alley, Dale Mitchell and Mickey Postman were going around to the 30,000 fire hydrants on the plant ground (ok. maybe not that many, but we did have a lot of them), and they were opening up the valves and flushing them out. That means, they were letting them run for a while to clear them out from any contaminates that may have built up over the year of not being used.
Throughout their adventure they had opened a multitude of Hydrants situated out in the fields along the long belt conveyor from the coalyard and around the two one-million gallon #2 Diesel tanks.
Large Oil Tank about the size of the two that are at the Power Plant
The brave Power Plant Men, learned that when opening a fire hydrant wide open in the middle of field had unintended consequences. It tended to wash out the ground in front of the flow of the water shooting out of the hydrant. So the team of experts devised a plan to place a board in front of the hydrant when it would be in danger of tearing a hole in the terrain. The board would divert the water into the air where it would fan out and not cause damage to the surrounding area.
This was working fine, and when they arrived at the fire hydrant next to the substation, since the stream from the hydrant was pointing directly into the substation (hmm. a design flaw, I think), they decided to prop the board up against the fence to keep from washing away the gravel in the substation. Well. When a fire hydrant is opened that hasn’t been used for a year, the first flow of water to shoot out is dark brown.
You may think that this is because the water has somehow become dirty over the past year, but that isn’t quite the case. What has happened is that the pipe has been rusting little by little and the water has become saturated with the rust. So, the water shooting out of the hydrant was full of rust (hence the need to flush them out).
Well. Rust is made of metal. Metal is conductive, especially when it is mixed with water. When the water hit the board, it was deflected into the air and happened to direct itself directly into the high voltage switch in the substation. This caused a circuit to the ground which, once it created an arc pumped all the electricity directly into the ground.
Normally when something like this happens it doesn’t trip the Main Power Transformer to a Power Plant.
A Main Power Transformer
This time it did. I know there was a few heads scratching trying to figure it out. I think I figured out what happened a little while later. You see… here is the rest of the story….
Once the unit was back online and the emergency was over, someone finally noticed that the telephone system couldn’t call outside of the plant. Well. I was the main telephone person at the time, so the control room called me and asked me to look into the problem.
I checked the telephone computer and it was up and running just fine. Internal calls could be made. Only any call outside just concluded with a funny humming sound. After checking the circuit in the Logic Room next to the Rolm Telephone Computer I headed for…. guess where….. the Main Switchgear….
A ROLM Phone Computer
In the middle of the main switchgear in the back of the room right next to the Auxiliary Substation beyond the back wall, the outside telephone line came into the plant. The first thing it did was go through a special Telephone Surge Protector.
Telephone Grounding Panel
In this picture above, the silver circular buttons on the left side are really an old style surge protector. whenever there was a power surge, the carbon connection in the surge protector would quickly melt causing the circuit to go straight to ground. Thus protecting the rest of the telephone circuit. So, if some kid in their house decides to connect the 120 volts circuit to the telephone for fun to see what would happen, this circuit would protect the rest of the phone circuits. Keep in mind that this was during the early 1990 when “Surge Protection” still was basically all “mechanical”.
Anyway, when I arrived at this panel and I checked the surge protector to the main line going out of the plant, guess what I found…. Yep. Shorted to ground. Luckily there were some spares that were not wired to anything in the panel and I was able to swap them out for the ones that had been destroyed. — These were a one time use. Which meant, if they ever had to short to ground, they had to be replaced.
Ok. Fine. After a little while, we were able to call back out of the plant, though there was still some residual noise on the line. It was like this… when you called out of the plant, the person on the other end sounded like they were buried in a grave somewhere and they were trying to talk to someone living just like in an episode of the Twilight Episode where a phone line landed on a grave and the dead person tried to call his long lost love from the past.
Gladys Cooper in the Twilight Zone Episode “Night Call”
I didn’t give it much thought other than that I figured the 189,000 volt arc to ground must have shorted out the telephone line since the phone line ran directly under the auxiliary substation ground grid.
It wasn’t until the next morning when the Southwestern Bell repairman showed up at the plant. I knew him well, since he had been working on our phone lines since before the AT&T breakup in 1984. When I met him in the front of the electric shop, he said that he needed to check our telephone circuits. I told him that I knew that we had a problem because we had a high voltage short to ground yesterday and I found our surge protectors melted away.
He explained to me that not only was our circuit affected, but that every relay house from here to Ponca City was blown out. That’s when I realized that the problem was the reverse of the usual situation. What had happened was that the Ground Grid in the substation and the surrounding area (including the Unit 1 Main Power Transformer) had become hot. What do you do when the ground grid becomes charged?
The Ground Grid is what is supposed to protect you when a surge happens, but what happens when the ground grid itself is the problem? In this case, when the high voltage line about 60 feet from the telephone cable surge protector, arced to ground, it fed a tremendous amount of power back through the ground grid. when equipment detected the surge in voltage, they automatically defaulted their circuits to ground. That’s why the telephone circuit died. That’s what tripped the Main Power Transformer.
When the telephone circuit detected the high voltage surge, it shorted to ground (which was the problem), causing the high voltage to feed directly into the phone line and down the line to the next Southwestern Bell relay switch, which also defaulted to ground, trying to bleed off the surge as it went from relay switch to switch until enough of the power was able to be diverted to ground.
That day sure turned out to be a learning experience. I learned that when all the lights go out in the control room, that it is almost assured that the coffee pot in the electric shop is going to stop working. I also learned that in order to coax the plant manager to the electric shop, a major electrical tragedy is one good way. I learned that when shooting rusty water into the air don’t point it at a high voltage auxiliary substation switch. — I’m sure Mickey Postman learned that lesson too. I also learned that just like in Star Trek… whenever there is a dangerous job to do, the Captain is always the one that wants to do it. Does that make sense? Send a Peon like me in there…
I also learned something else about Power Plant Men…. You see…. People like Dale Mitchell, George Alley and Mickey Postman all are examples of incredibly wonderful Power Plant Men. When they were out there doing their duty and something tragic like this, all the Power Plant Men felt their pain. They knew that they all felt guilty for tripping the unit. It didn’t matter that a million dollars every so many minutes was walking out the door in revenue. The only thing that mattered was that these three men were safe.
Mickey Postman
Since I have left the Power Plant, I have found that the idea that the employee is the greatest asset that a company can possess is not a universal idea. You see, there was never the thought that any of these people should be fired for their mistake. On the contrary. The true Power Plant Men did whatever they could to let them know that they knew exactly how they felt. It could have happened to any of them.
Besides the friendship between Power Plant Men, one of the things I miss most about working at the Power Plant is that the employees are held in high esteem as a real asset to the company. Many could learn from their example.
Comments from the Original post
-
-
That was an exciting day! Another great story. Thanks for the memories.
-
-
-
I like the mix of storytelling and information sharing you deliver here. Thanks again
-
-
-
Aug 8, 2011 I lost all reserve in and lost both units in 2 separate storms and A1 would not start for a few minutes and A2 was leaking antifreeze terribly not a scratch and back online in 34 hours both units and no one ever asked me one question even with a Safety Dept not one question asked of me the SS on duty. Not many men on the planet have ever experienced an uncontrolled total plant outage- you would of thought a learning opportunity would of took place. Feb 2011 worst winter temps in years 50 Units or more tripped in Texas let alone the trusty units of Redbud and McClain were fighting Sooner rolled right along with storm warnings for two weeks ahead – the ICS still went to Detroit for just tours of other facilities once again the fall guy Tarver McArthur stood alone. I had authored a Freeze Protection Plan for the plant and that seemed to save the day and explain to the regulatory bodies how we were online and everyone else in Texas was off and enjoying rolling blackouts in a terrible winter weather situation not to mention our powers that be were all stranded in Detroit and very few people could get to the plant without getting stuck trying to get there as well – but a few health heart issues later I am still here to tell about it all you would think folks would want to take advantage of someone that had went through the fire and Ice but thats ok I want them someday distant to get all the credit they deserve when the trumpets sound as that is truly what matters most. DT
-
-
Great story, and yes, you took exactly the proper lesson from it, and it is too bad that many of our bosses haven’t learned it.
And yup, I’ve buried a lot of electrocuted squirrels over the years.
Psychological Profile of a Power Plant Control Room Operator
Originally posted on July 6, 2012:
I suppose that many parents while raising their children would hear them say, “Dad, can you read that story to us again about the pirates that go to the island to find the treasure but Jim Hawkins fights them single-handed?” Or their children might say to their mother, “Will you tell us the story again about how you met daddy?” In my household, my children would say, “Dad, tell us another story about how you played a joke on Gene Day when you were working at the Coal-Fired Power Plant in Oklahoma? Those are always the best!”
As I pointed out in my third post this year called “Power Plant Humor And Joking With Gene Day” the first time I met Gene Day, I could tell that he was the type of person that would take a joke well, or so I thought…. One of the favorite stories my daughter would like for me to tell her as she was growing up was the one where I had created a Psychological Profile of Gene Day, who at that time, an Auxiliary Operator.
It began one day when I was leaving the electric shop through the Turbine Generator (T-G) building ground floor. A very noisy location as large steam pipes wound around under the Turbines where the steam caused a rumbling or whining sound. It was normal when walking through this area to reach up to the ear plugs that were draped over your shoulders and put them in your ears because the decibels were dangerously high if you were exposed earplugless too long.
Earplugs on a cord that can be draped over your shoulders when not in use.
I stepped from the landing leading from the electric shop and started toward number 1 boiler when I spied Gene Day making his way around the first floor of the T-G building inspecting equipment and marking his documents indicating that they were operating correctly. As I saw him turn toward my direction I quickly dodged behind the nearest metal pillar (I-Beam). I peaked my head out from behind the pillar and took out the notepad that was in my back pocket, and the pen from my vest pocket pocket protector.
A pocket protector is a must for electricians and computer nerds who need a place to keep their small tools.
Gene saw me and gave me a suspicious look as I began feverishly writing in my notepad while looking from my notepad back to Gene and then back to what I was writing. After I had done this for about 10 seconds or so, I put my pad away, my pen back in the pocket protector and strolled away toward unit 1 to continue my work.
A notepad like this
It happened that this particular week was Gene’s week to monitor the equipment in the T-G building, so throughout the week he would be making his way somewhere around the T-G building with his clipboard in hand. Each time I encountered him I would do the same thing. I would visibly hide behind a beam and write notes in my notepad. I saw Gene rather frequently during the week because the majority of the time, I left the electric shop by going through the T-G room to one of the boilers and then to the precipitators, where I spent most of my time working at this time in my career (but that is another story for a later time).
Each time, Gene would watch me suspiciously knowing that I was just messing with him, but not exactly sure what I was “up to”. At the time, I wasn’t sure either. So I just wrote down what I saw Gene doing, that way, if he ran over and grabbed my notepad from me, it wouldn’t say anything other than what I saw.
It had happened at the plant a few years earlier when I was a janitor that the company had hired an efficiency expert to monitor the employees at the plant. He would walk around the plant with a stopwatch observing the employees. When he saw them he would write notes on his clipboard. It became very unnerving because you would walk around the corner and there he would be standing writing something down about you.
I went to the Assistant Plant Manager Bill Moler and told him that this creepy guy keeps showing up in the main switchgear by the janitors closet. And every time he sees me, he writes something down. I told Bill that it really bothered me. He explained that he is an efficiency expert and he has a certain path that he takes throughout the day and takes a snapshot of what the workers are doing at that moment and writes it down. By doing that he calculates how efficient we are. It seemed pretty silly to me, because most mechanics when they saw him coming put their tools down and did nothing while he walked by until he was out of sight again.
When I was on the labor crew a few weeks later, and I was blowing coal dust off of all the I-Beams above the bowl mills with a high pressure air hose, I looked down, and through my fogged up goggles I could see this guy standing directly under me. I was about 50 feet above him crawling across an I-Beam with the air hose blowing black dust everywhere. He had crossed my barrier tape to go into the bowl mill area to see how efficient I was being.
This is the type of barrier tape I was using. It is made of woven plastic fibers.
I was so mad I turned off my air hose, climbed down the wall Spiderman-like (no. not head first) and went straight into the A-Foreman’s office and told Marlin McDaniel that the Efficiency Expert had crossed my barrier tape and was standing directly under me as I was blowing down the beams with air.
It turned out that the Efficiency Expert had gone upstairs to complain that some guy in the bowl mill had dumped a bunch of coal dust on him while he was monitoring him from below. Evidently, he wasn’t expert enough to know you weren’t supposed to cross someone’s barrier tape without permission, as was indicated on the caution tags that were tied to the barrier tape. From that point on, the efficiency expert (now in lowercase) had to be accompanied by someone from the plant to make sure he wasn’t breaking any safety rules and putting himself at harm.
To make a long side story short, we turned out to be so efficient, people came from all over the world to study us. Somebody downtown hired the efficiency expert full time, but later he was laid off during the first downsizing.
He reminds me of a person on an episode of Star Trek The Next Generation where this guy Lieutenant Commander Remmick comes aboard the Enterprise and he walks around inspecting everything and asking everyone questions that make them uncomfortable and at the end asks Picard if he could come work on the Enterprise. He looked so much like him, that I thought maybe our efficiency expert went to Hollywood to become an actor.
Lieutenant Commander Remmick
Back to Gene Day. I suppose the thought of the efficiency expert may have been going through my mind as I was taking notes about Gene Day at work. Like I said, at the time, I didn’t know what I was going to do with it.
It finally came to me on Thursday morning. This was the last day that Gene was going to be on the day shift, so I figured I might as well do something about it. So when I entered the shop that morning, I sat down at the desk of my foreman Andy Tubbs and began to write. The title at the top of the page was: “The Psychological Profile of Gene Day”.
Using the notes I had taken during the week, I wrote things like the following: “Gene Day walks around the Turbine Generator building with a clipboard in his hand trying desperately to look like he’s doing something important. He constantly hopes that someone is watching him because he dislikes doing so much work to act busy for no reason.
At times Gene Day gets paranoid and believes that he sees people spying on him from behind every corner (especially I-Beams). Sometimes Gene Day stands in the middle of the T-G floor staring up into space as if he forgot what he was supposed to be doing.” I’m sure I wrote more, but I don’t remember everything… but I do remember the second from the last sentence. I will save that for a couple of paragraphs from now.
I took my Psychological Profile of Gene Day and went up to the Control Room. I noticed out of the corner of my eye that Gene had just walked over toward the break room behind the Auxiliary Control Panel so I walked over by the Shift Supervisor’s office and I leaned against the top of the large Blue Monitor and placed the Psychological Profile on the top of the monitor in front of me.
Less than a minute later Gene came walking around behind me and seeing the paper on the top of the monitor came up behind me and looked over my shoulder and began to read….. He wasn’t trying to hide the fact that he was reading the paper, and I obviously knew he was there, and the title in large bold letters did say, “The Psychological Profile of Gene Day”. So, he read on.
I heard a few chuckles as he read through my interpretation of what he had done during the week. Then he came to the “Second from the Last Sentence”, as I could hear him reading quietly in my ear… The sentence read, “Gene Day sneaks up behind people and reads their private material over their shoulder!” — Bingo! I had him!
When he read that he grabbed me by the throat and started to Throttle me! Shaking me back and forth. This would have been a humdinger of a joke at that point, but I had one more sentence up my sleeve… ur… I mean on the paper….
As I was waivering (is that a word?) back and forth between life and death I managed to eke out something like: “Wait! There’s More!”…. Gene Day let up on me a little and looked down at the page and read the last sentence…… It read….. “Gene Day tries to strangle people who are only trying to help him by creating his Psychological Profile.”
That was all it took. Another perfect joke played on Gene Day, and I was able to live to tell about it. When Gene read that he was stunned into dismay. Giggling as hard as he could he retreated shaking his head in defeat.
Now, I know that Gene reads these posts, and he may remember this story a little differently, and I’ll give him that because Gene is older than dirt and his memory isn’t that good. But I was the one that was being strangled, and I still have a vivid image of those few moments. Not only do I, but so do my children, who will one day tell their children, who will say, “Mommy, will you tell us that story again about how Grandpa was strangled by Gene Day that time in the Control Room at the Coal-Fired Power Plant in Oklahoma?”
Runaway Fire Hydrant Leaves Power Plant in the Dark
Originally posted May 17, 2014:
Don’t believe it when the Electric Company tells you that the reason your town lost electricity for an hour was because a squirrel climbed onto a transformer and shorted it out. The real reason just may be more bizarre than that and the company doesn’t want you to know all the different creative ways that power can be shut off. This is a tale of just one of those ways. So, get out your pencil and paper and take notes.
Power Plant Notepad
One spring day in 1993 while sitting at the Precipitator computer for Unit one at the Coal-fired Power Plant in North Central Oklahoma, while I was checking the controls to make sure all the cabinets were operating correctly, suddenly there was a distant boom, and the lights in the control room went out. The computer stayed on because it was connected to an electric panel called the VSP or Vital Services Panel, which in turn was supplied by the UPS system (Uninterruptible Power Supply). That was one of those moments where you may pause for a moment to make sure you aren’t still at home dreaming before you fly into a panic.
The Precipitator cabinets all indicated on the computer that they had just shutdown. I rose from the chair and walked around to the front of the Alarm Panel for Unit one, and found that the fluorescent lights were only out on Unit 1. The lights were still on for Unit 2. The Control Panel was lit up like a Christmas Tree with Green, Red, Blue and Yellow Lights. The Alarm Printer was spewing out paper at high speed. As the large sheets of paper were pouring out onto the floor, I watched as Pat Quiring and other brave Power Plant Control Room operators were scurrying back and forth turning switch handles, pushing buttons, and checking pressure gauges.
Just this site alone gave me confidence that everything was going to be all right. These Control Room operators were all well trained for emergencies just like this, and each person knew what their job was. No one was panicking. Everyone was concentrating on the task at hand.
Someone told me that we lost Unit 1, and the Auxiliary Power to Unit 1 at the same time. So, Unit 1 was dead in the water. This meant, no fans, no pumps, no lights, no vending machines, no cold water at the water fountain and most importantly, no hot coffee!!! I could hear steam valves on the T-G floor banging open and the loud sound of steam escaping.
I turned quickly to go to the electric shop to see what I could do there in case I was needed. I bolted out the door and down the six flights of stairs to the Turbine-Generator (T-G) basement. Exiting the stairway, and entering the T-G basement the sound was deafening. I grabbed the earplugs that were dangling around my neck and crammed them into my ears. Steam was pouring out of various pop-off valves. I ducked into the electric shop where across the room Andy Tubbs, one of the electric foreman was pulling large sheets of electric blueprints from the print cabinet and laying them across the work table that doubled as the lunch table.
Andy Tubbs – True Power Plant Electrician
When I asked Andy what happened, I learned that somehow when a crew was flushing out a fire hydrant the water somehow shot up and into the bus work in the Auxiliary Substation (that supplies backup power to the Power Plant) and it shorted out the 189,000 volt substation directly to ground. When that happened it tripped unit 1 and the auxiliary substation at the same time leaving it without power.
An example of part of an Auxiliary Substation
I will explain how a fire hydrant could possibly spray the bus work in a substation in a little while, but first let me tell you what this meant at the moment to not have any power for a Power Plant Boiler and Turbine Generator that has just tripped when it was at full load which was around 515 Megawatts of power at the time.
Normally when a unit trips, the boiler cools down as the large Force Draft (FD) Fans blow air through the boiler while the even larger Induced Draft (ID) fans suck the air from the boiler on the other end and blow the hot air up the smoke stack. This causes the steam in the boiler tubes to condense back into water. Steam valves open on the boiler that allow excessive steam to escape.
When the boiler is running there is a large orange fireball hovering in space in the middle of the boiler. The boiler water is being circulated through the boiler and the Boiler Feed Pump Turbines are pumping steam back and forth between the turbine generator and the boiler reheating the steam until every bit of heat from the boiler that can be safely harnessed is used.
When all this stop suddenly, then it is important that the large fans keep running to cool down the steam, since it is no longer losing energy in the generator as it was when it was busy supplying electricity to 1/2 million people in Oklahoma City. The power is fed to the fans from the Auxiliary substation located right outside the Main Switchgear where all the breakers reside that supply the power to the fans. Unfortunately, in this case, the Auxiliary substation was shutdown as well, leaving the boiler without any fans.
Without fans for cooling, and pumps to circulate the water, the walls of the boiler began heating up to dangerous temperatures. Steam was whistling out of pop off valves, but if the steam drum on the top of the boiler were to run dry, then the entire boiler structure could be compromised and begin melting down. — So, this was serious. Something had to be done right away. It wouldn’t be as bad as the China Syndrome since we were burning coal instead of nuclear power, but it would have caused a lot of damage nonetheless.
From the movie “The China Syndrome” where a similar emergency existed only in the movie, it was a Nuclear Plant
I have a side story about this picture, but I think I’ll save it for another post because I don’t want to digress from the main story at this point (Ok. Let me just say “Jack Maloy and Merl Wright” for those who can’t wait) See the post: “Power Plant Conspiracy Theory“.
With the prospect that the boiler might melt to the ground in a pile of rubble, it would seem that the main priority was to turn the Auxiliary Substation back on so the fans could be turned back on and prevent the boiler from collapsing. So, we walked out to the substation and looked at the switches that would have to be operated in order to first power up the main bus and then to close to supply power to the two big transformers and the six smaller transformers that supplied the Unit 1 Main Switchgear.
While inspecting the switches where the electricity had gone to ground we found that one of the main insulators was cracked.
A High Voltage Insulator like this
Since this insulator was cracked, we didn’t really want to operate the switch to test if another 189,000 volts would go straight to ground again, especially since one of us would be standing right underneath it cranking the switch. So, we went back to the shop to find an alternative.
By this time the Plant Manager, Ron Kilman arrived in the shop, and understanding the urgency to find a solution asked us what were the alternatives. He was relying on our expertise to make the decision.
Ron Kilman is the one on the left in the plaid shirt
The other solution would be to cut the power over from Unit 2 which was still humming away pushing electricity to Oklahoma City out of the 345,000 volt substation. The cut over would be very simple because the switchgear was designed with this in mind. We analyzed the power rating on the auxiliary transformers on Unit 2 and thought that we might be cutting it close to have them running both sets of fans at the same time, especially since the full load amps of a huge fan starting up was about 10 times the normal rate.
The transformer was rated to handle the load, but consider this. What if this caused Unit 2 to trip as well. With the Auxiliary substation offline, if Unit 2 tripped, we would be in twice the amount of trouble we were currently in. What a day it would have been if that had happened and two 250 foot boilers had come crashing to the ground in a pile of rubble. After reading the power ratings on the auxiliary transformers I was thinking, “Yeah, let’s do it! These transformers can handle it.” Andy was not so eager.
So, we were left with one alternative. That was to shut the switch in the Auxiliary substation that had the cracked insulator and take our chances that it wasn’t going to short to ground and blow up over our heads. I think I was eager to close the switch for Andy, but if I remember correctly, he didn’t want me to be the one to suffer the consequences and decided to close the switch himself. Needless to say. Andy closed the switch, and nothing blew up.
High Voltage Switch similar to the one we were closing
As soon as the power was restored to the switchgear, the fans were powered up and the temperature in the boiler was quickly reduced. The coffee pot in the Electric Shop began heating the coffee again. The power plant was saved from a major catastrophe. That was delayed for another day… of which I will talk about later (see the post “Destruction of a Power Plant God).”
So, how exactly does a fire hydrant shoot water up into the bus work of a substation like the picture of the switch directly above? The culprit fire hydrant wasn’t in the substation, it sat alongside it outside the fence a good 50 feet from the high voltage switch. No hose was attached to the fire hydrant. It was only being flushed out as part of a yearly activity to go around and make sure the fire hydrants are all operating correctly.
Here is the story about how the squirrel climbed into the transformer this time….
George Alley, Dale Mitchell and Mickey Postman were going around to the 30,000 fire hydrants on the plant ground (ok. maybe not that many, but we did have a lot of them), and they were opening up the valves and flushing them out. That means, they were letting them run for a while to clear them out from any contaminates that may have built up over the year of not being used.
Throughout their adventure they had opened a multitude of Hydrants situated out in the fields along the long belt conveyor from the coalyard and around the two one-million gallon #2 Diesel tanks.
Large Oil Tank about the size of the two that are at the Power Plant
The brave Power Plant Men, learned that when opening a fire hydrant wide open in the middle of field had unintended consequences. It tended to wash out the ground in front of the flow of the water shooting out of the hydrant. So the team of experts devised a plan to place a board in front of the hydrant when it would be in danger of tearing a hole in the terrain. The board would divert the water into the air where it would fan out and not cause damage to the surrounding area.
This was working fine, and when they arrived at the fire hydrant next to the substation, since the stream from the hydrant was pointing directly into the substation (hmm. a design flaw, I think), they decided to prop the board up against the fence to keep from washing away the gravel in the substation. Well. When a fire hydrant is opened that hasn’t been used for a year, the first flow of water to shoot out is dark brown.
You may think that this is because the water has somehow become dirty over the past year, but that isn’t quite the case. What has happened is that the pipe has been rusting little by little and the water has become saturated with the rust. So, the water shooting out of the hydrant was full of rust (hence the need to flush them out).
Well. Rust is made of metal. Metal is conductive, especially when it is mixed with water. When the water hit the board, it was deflected into the air and happened to direct itself directly into the high voltage switch in the substation. This caused a circuit to the ground which, once it created an arc pumped all the electricity directly into the ground.
Normally when something like this happens it doesn’t trip the Main Power Transformer to a Power Plant.
A Main Power Transformer
This time it did. I know there was a few heads scratching trying to figure it out. I think I figured out what happened a little while later. You see… here is the rest of the story….
Once the unit was back online and the emergency was over, someone finally noticed that the telephone system couldn’t call outside of the plant. Well. I was the main telephone person at the time, so the control room called me and asked me to look into the problem.
I checked the telephone computer and it was up and running just fine. Internal calls could be made. Only any call outside just concluded with a funny humming sound. After checking the circuit in the Logic Room next to the Rolm Telephone Computer I headed for…. guess where….. the Main Switchgear….
A ROLM Phone Computer
In the middle of the main switchgear in the back of the room right next to the Auxiliary Substation beyond the back wall, the outside telephone line came into the plant. The first thing it did was go through a special Telephone Surge Protector.
Telephone Grounding Panel
In this picture above, the silver circular buttons on the left side are really an old style surge protector. whenever there was a power surge, the carbon connection in the surge protector would quickly melt causing the circuit to go straight to ground. Thus protecting the rest of the telephone circuit. So, if some kid in their house decides to connect the 120 volts circuit to the telephone for fun to see what would happen, this circuit would protect the rest of the phone circuits. Keep in mind that this was during the early 1990 when “Surge Protection” still was basically was all “mechanical”.
Anyway, when I arrived at this panel and I checked the surge protector to the main line going out of the plant, guess what I found…. Yep. Shorted to ground. Luckily there were some spares that were not wired to anything in the panel and I was able to swap them out for the ones that had been destroyed. — These were a one time use. Which meant, if they ever had to short to ground, they had to be replaced.
Ok. Fine. After a little while, we were able to call back out of the plant, though there was still some residual noise on the line. It was like this… when you called out of the plant, the person on the other end sounded like they were buried in a grave somewhere and they were trying to talk to someone living just like in an episode of the Twilight Episode where a phone line landed on a grave and the dead person tried to call his long lost love from the past.
Gladys Cooper in the Twilight Zone Episode “Night Call”
I didn’t give it much thought other than that I figured the 189,000 volt arc to ground must have shorted out the telephone line since the phone line ran directly under the auxiliary substation ground grid.
It wasn’t until the next morning when the Southwestern Bell repairman showed up at the plant. I knew him well, since he had been working on our phone lines since before the AT&T breakup in 1984. When I met him in the front of the electric shop, he said that he needed to check our telephone circuits. I told him that I knew that we had a problem because we had a high voltage short to ground yesterday and I found our surge protectors melted away.
He explained to me that not only was our circuit affected, but that every relay house from here to Ponca City was blown out. That’s when I realized that the problem was the reverse of the usual situation. What had happened was that the Ground Grid in the substation and the surrounding area (including the Unit 1 Main Power Transformer) had become hot. What do you do when the ground grid becomes charged?
The Ground Grid is what is supposed to protect you when a surge happens, but what happens when the ground grid itself is the problem? In this case, when the high voltage line about 60 feet from the telephone cable surge protector, arced to ground, it fed a tremendous amount of power back through the ground grid. when equipment detected the surge in voltage, they automatically defaulted their circuits to ground. That’s why the telephone circuit died. That’s what tripped the Main Power Transformer.
When the telephone circuit detected the high voltage surge, it shorted to ground (which was the problem), causing the high voltage to feed directly into the phone line and down the line to the next Southwestern Bell relay switch, which also defaulted to ground, trying to bleed off the surge as it went from relay switch to switch until enough of the power was able to be diverted to ground.
That day sure turned out to be a learning experience. I learned that when all the lights go out in the control room, that it is almost assured that the coffee pot in the electric shop is going to stop working. I also learned that in order to coax the plant manager to the electric shop, a major electrical tragedy is one good way. I learned that when shooting rusty water into the air don’t point it at a high voltage auxiliary substation switch. — I’m sure Mickey Postman learned that lesson too. I also learned that just like in Star Trek… whenever there is a dangerous job to do, the Captain is always the one that wants to do it. Does that make sense? Send a Peon like me in there…
I also learned something else about Power Plant Men…. You see…. People like Dale Mitchell, George Alley and Mickey Postman all are examples of incredibly wonderful Power Plant Men. When they were out there doing their duty and something tragic like this, all the Power Plant Men felt their pain. They knew that they all felt guilty for tripping the unit. It didn’t matter that a million dollars every so many minutes was walking out the door in revenue. The only thing that mattered was that these three men were safe.
Mickey Postman
Since I have left the Power Plant, I have found that the idea that the employee is the greatest asset that a company can possess is not a universal idea. You see, there was never the thought that any of these people should be fired for their mistake. On the contrary. The true Power Plant Men did whatever they could to let them know that they knew exactly how they felt. It could have happened to any of them.
Besides the friendship between Power Plant Men, one of the things I miss most about working at the Power Plant is that the employees are held in high esteem as a real asset to the company. Many could learn from their example.
Comments from the Original post
-
That was an exciting day! Another great story. Thanks for the memories.
-
I like the mix of storytelling and information sharing you deliver here. Thanks again
-
Aug 8, 2011 I lost all reserve in and lost both units in 2 separate storms and A1 would not start for a few minutes and A2 was leaking antifreeze terribly not a scratch and back online in 34 hours both units and no one ever asked me one question even with a Safety Dept not one question asked of me the SS on duty. Not many men on the planet have ever experienced an uncontrolled total plant outage- you would of thought a learning opportunity would of took place. Feb 2011 worst winter temps in years 50 Units or more tripped in Texas let alone the trusty units of Redbud and McClain were fighting Sooner rolled right along with storm warnings for two weeks ahead – the ICS still went to Detroit for just tours of other facilities once again the fall guy Tarver McArthur stood alone. I had authored a Freeze Protection Plan for the plant and that seemed to save the day and explain to the regulatory bodies how we were online and everyone else in Texas was off and enjoying rolling blackouts in a terrible winter weather situation not to mention our powers that be were all stranded in Detroit and very few people could get to the plant without getting stuck trying to get there as well – but a few health heart issues later I am still here to tell about it all you would think folks would want to take advantage of someone that had went through the fire and Ice but thats ok I want them someday distant to get all the credit they deserve when the trumpets sound as that is truly what matters most. DT
-
Great story, and yes, you took exactly the proper lesson from it, and it is too bad that many of our bosses haven’t learned it.
And yup, I’ve buried a lot of electrocuted squirrels over the years.
Resistance in a Coal-Fired Power Plant
Originally posted April 19, 2013:
Resistance is Futile! You may have heard that before. Especially if you are a Star Trek Fan. If not, then you know that there is always some form of resistance wherever you are.
Captain Picard as Locutus trying to convince you that “Resistance is Futile” Like that is ever going to happen
I learned a lot about resistance when I first joined the electric shop at the coal-fired power plant in North Central Oklahoma in 1984. I was assigned to work with Sonny Kendrick and Bill Rivers on the Precipitator during overhauls and when I wasn’t working on the manhole pumps and there wasn’t any other emergencies going on. Actually, from 1984 on, on, the Precipitator for the next 17 years I continued to work on the precipitator… (if I had only known my fate….).
Not only did I learn a lot about resistance, I also learned about capacitance, reactance, transformers, rectifiers, power supplies, diodes, transistors, op amps, and pots (also known as potentiometers). Bill Rivers was the brains of the outfit. Sonny was the Electric Specialist banished to the Precipitator by Leroy Godfrey (See Singing Along with Sonny Kendrick). Bill thought up the ideas and Sonny went to work to implement them. I just jumped in where I was needed.
The Precipitator is the large box between the boiler and the smokestack (maybe you can see this in the Power plant picture). The purpose of the electrostatic precipitator is to take the smoke (or fly ash) out of the exhaust before it went out of the smokestack. The controls for the Precipitator were all electronic at that time. That meant that there were circuit boards full of resistors, capacitors, transistors, operational amplifiers, diodes and potentiometers. These circuit boards controlled the way the power was distributed throughout the precipitator wires and plates through high powered transformers, and how the rappers and vibrators operated that dropped the collected ash into the hoppers.
Bill had me take an electronics course at the Indian Meridian Vo-tech so I would know the basics. Then he taught me all the shortcuts. I had to be able to look at a resistor and tell right away what the value of resistance it was. Resistors are color-coded and you had to learn what each of the colors represented…
This for instance is a 339 ohm resistor with a 5% tolerance.
I was expected to know this by sight. Bill would test me. There was a mnemonic device that I was taught to remember what each color represented, but it is not appropriate to repeat it, so I won’t. It is enough to say that the colors go like this: Black, Brown, Red, Orange, Yellow, Green, Blue, Violet, Gray, White (I will never forget this my entire life). These represent the numbers zero through 9. Here is a full explanation of how to read a resistor….. just in case you are curious, or you are such a boring person that you really need some material to bring up when you are at a party and don’t know what to say:
This is how the color codes work for resistors works
I found that having just the correct amount of resistance was very important. Too much or too less, and everything stops working.
Isn’t it that way with management also? If the management is too resistant to change, then things come to a halt. If they have too little resistance, they lose control of the situation. Depending on the circuit (or managerial decision) and what you are trying to do, it helps to have a manager that has a variable resistance to meet the needs of each situation. Resistance to change is always a balancing act.
During the first two years I was an electrician, the main control panels that controlled the operation of the precipitators were electronic. We spent a lot of time in the lab troubleshooting electric circuits looking for blown (or bad) parts that needed to be replaced. Then we would solder new components on the circuit boards and then put them back in operation. I learned how to be an electronics junky. I became addicted to fixing electronic circuit boards. It was like a game to me.
Later, the precipitator controls were changed to digital controls. That is, they were more like little computers controlling the precipitator. Instead of a bunch of circuit boards dumbly, but cleverly, doing their job, (how many commas can I use in one sentence?), little brains were added that made decisions and reacted to conditions in a much more dynamic way.
What was interesting was that one day Bill Rivers was describing how technology was going to be in the future. He said that some day, we will be able to sit in the lab and look on a computer and see what all the controls in the precipitator were doing (this was 1984). If something isn’t working right, we could just reach over, type a few keys on the computer and adjust the controls. Drink our sweetened tea (a necessary staple in Oklahoma at the time), and then wait for the next crisis…. Then he would giggle at the look of disbelief on my face.
When he was telling me this, I was thinking in my head…. Well, that would be nice, but this sounds more like a pipe dream to me than reality. What does an older guy with six kids from a tool and die company in Columbia Missouri (where I grew up, by chance) know about the future of anything….. well…. anything…uh… new age…. If that is what you might call it… I found out you just don’t really know when you are sitting in front of a true “visionary” with tremendous insight.
Bill Rivers had this incredible knack for telling the future. In 1984 he was predicting computer controls in the control room where you ran the entire plant from a computer on a desk instead of using the “Big Board”.
I love this picture! It makes me feel at home! This was not our plant, but is a Power Plant control room
He said you would be able to call someone on a phone you kept in your pocket or your watch like Dick Tracey.
Dick Tracy in a control room of a power plant sending commands to the hopper nozzle booster pumps
I don’t know what journals Bill was reading or if he just dreamed all this stuff up in his head, or maybe he was a Star Trek Fan that believed that if you can dream it up you can do it. I do know that he picked up on subtle queues and made great inferences from them that seemed astronomically unlikely. However, I have to admit that he caught me off guard a number of times with predictions that definitely came true.
I will talk about this more in a future post, but for now I will say that we did upgrade the precipitator to where you could sit in the control room and monitor and adjust the precipitator controls (all 84 on each unit), and even each of the rappers (672 rappers) and vibrators (168 vibrators) on the roof of each precipitator. With one key on the computer I could send a plume of ash out of the smokestack that looked like the unit had just tripped, and a moment later, clean it up again. This meant that I could send smoke signals to the Osage Indian tribe 20 miles north up the Arkansas (pronounced “Are Kansas”) river, telling them that the Pow Wow would begin at sunset.
Today, I understand that the “Big Board” at the plant is just a large junction box and the plant is controlled almost (if not) completely by computers sitting on the desk. Before I left the plant in 2001, this was being transitioned slowly to computer controls. I have another story to tell some day about this, and how an operator named Jim Cave, a Power Plant Genius and true Power Plant Man of the highest integrity, was snubbed by upper management for speeding this technology along. — Another example of Power Plant Resistance….
But for now…. back to my electronic days… before I began re-programming the Eeprom chips in the precipitator controls….
An Eeprom Chip used in the precipitator controls which I had a very “personal” relationship with…
Bill Rivers confided with me one day that when the new Instrument and Controls department had been formed from the “Results” department that his dream had been to become a part of this team. It meant the world to him. It was where he believed he belonged. It was one of his major goals in life.
There used to be two electrical specialists in the Power Plant. Sonny Kendrick was not always the only one. The other specialist was chosen to go to the Instrument and Controls shop. Bill Rivers wanted to move there also. He definitely had the experience and the knowledge to be a superb instrument and controls person. But Bill had this one problem.
He loved to joke around. He loved to pull strings and push buttons. I have mentioned in a previous posts that Bill would play a new joke on Sonny Kendrick every single day. As I have unfortunately found out in my own life… this tends to make them…. well….. it tends to make enemies out of those who have a chip on their shoulder. Those people who naturally feel inadequate in their abilities or their position in life. To go one step further…. anyone who feels “unloved”….. these people definitely do not like being joked with. They seem to never forgive you. My greatest regret in life is joking around with these individuals.
So, when it came time to choose who would be a part of the new Instrument and Controls shop, Bill Rivers was turned down. It was explained to him that the reason he was not given the job was because he cut off the leads of a resistor when he replaced them. — I’m not kidding. Bill Rivers had the habit of cutting off the leads of each resistor, transistor, diode or capacitor that he replaced…. this is why Monty Adams turned down his request for joining the “elite” Instrument and Controls shop (as he told Bill to his face).
Someone had told the Instrument and Controls Supervisor Monty Adams that Bill Rivers cut the leads off of transistors and resistors when he replaced them so that you couldn’t test them to see if they were all right. Implying that he didn’t want you to know whether he had replaced the transistor or resistor by mistake.
Bill Rivers took several transistors, cut the leads off of each of them and handed them to me and asked me to test them to see if they was still good or if they were bad. I took out my voltmeter, set it to ohms, and proceeded to test them as Bill Rivers had taught me. I told him…. this transistor is good….. this one is bad….
You see…. there is no way to cut the leads off of a transistor in such a way to make it impossible to tell if a transistor is good or bad…. In reality…. you cut the leads off of a bad transistor so that the person working on the circuit board knows that this is a bad transistor and doesn’t use it again by accident. This was electronics 101.
When Bill told me this story, he literally had tears in his eyes. This was because being part of the Instrument and Controls team was part of his dream. His family and the entire rest of his life was decided the day he was told that he was not going to be a part of a team that he believed was his true lot in life.
I remember his exact words as he sat there in the lab alone and told me this story. He said, “… and Monty didn’t know… He didn’t know that you cut the leads…. that is standard procedure….” In Bill’s giggly way, he was crying out loud as he told me this.
From that point on….I knew that the decisions Bill made in his life were driven by that one decision to exclude him from this team. Unlike many of us that could say to ourselves…. “That is their loss”…. Bill kept this pain in his heart each day…. Every decision from that day further was effected by this event.
I calculated it out one day that I spent 414 hours driving back and forth from Stillwater, Oklahoma to the plant and back each day with Bill Rivers (along with Yvonne Taylor and Rich Litzer and occasionally others that needed a ride), and over that time, I became very close to Bill, even to the point of tutoring his son in Algebra (see post: How Many Power Plant Men Can You Put in a 1982 Honda Civic?).
I say this because I know about the pain that inflicted Bill River by a rash decision based on the hearsay of someone that held a grudge. I know how his entire life was changed and how it ended for Bill Rivers as a power plant employee. I know that every decision by Bill after this date was made in response to this one decision. Anyone who experienced Bill after 1983 knows what I am talking about.
I realized that today my own decisions in life help spell out my future. How some little remark may be misinterpreted, or even properly so. I realize as I write this post that how I accept or reject these events in my life, determines the future of my family. After seeing how every event in Bill’s life after that day at the power company was determined by his experience was to his detriment, I am determined not to let the same thing happened to me…..
That is why I have taken on the philosophy in my life that no matter how my actions are misinterpreted, I am determined to remain true to myself. I know what I mean, and I mean what I say, and I say what I mean, and an Elephant is Faithful 100 %.
From Horton Hatches an Egg by Dr. Seuss
Comment from the Original Post:
-
It’s amazing how many decisions are made based on incorrect / incomplete information (at all levels).
Psychological Profile of a Power Plant Control Room Operator — Repost
Originally posted on July 6, 2012:
I suppose that many parents while raising their children would hear them say, “Dad, can you read that story to us again about the pirates that go to the island to find the treasure but Jim Hawkins fights them single-handed?” Or their children might say to their mother, “Will you tell us the story again about how you met daddy?” In my household, my children would say, “Dad, tell us another story about how you played a joke on Gene Day when you were working at the Coal-Fired Power Plant in Oklahoma? Those are always the best!”
As I pointed out in my third post this year called “Power Plant Humor And Joking With Gene Day” the first time I met Gene Day, I could tell that he was the type of person that would take a joke well, or so I thought…. One of the favorite stories my daughter would like for me to tell her as she was growing up was the one where I had created a Psychological Profile of Gene Day, who at that time, an Auxiliary Operator.
It began one day when I was leaving the electric shop through the Turbine Generator (T-G) building ground floor. A very noisy location as large steam pipes wound around under the Turbines where the steam caused a rumbling or whining sound. It was normal when walking through this area to reach up to the ear plugs that were draped over your shoulders and put them in your ears because the decibels were dangerously high if you were exposed earplugless too long.
Earplugs on a cord that can be draped over your shoulders when not in use.
I stepped from the landing leading from the electric shop and started toward number 1 boiler when I spied Gene Day making his way around the first floor of the T-G building inspecting equipment and marking his documents indicating that they were operating correctly. As I saw him turn toward my direction I quickly dodged behind the nearest metal pillar (I-Beam). I peaked my head out from behind the pillar and took out the notepad that was in my back pocket, and the pen from my vest pocket pocket protector.
A pocket protector is a must for electricians and computer nerds who need a place to keep their small tools.
Gene saw me and gave me a suspicious look as I began feverishly writing in my notepad while looking from my notepad back to Gene and then back to what I was writing. After I had done this for about 10 seconds or so, I put my pad away, my pen back in the pocket protector and strolled away toward unit 1 to continue my work.
A notepad like this
It happened that this particular week was Gene’s week to monitor the equipment in the T-G building, so throughout the week he would be making his way somewhere around the T-G building with his clipboard in hand. Each time I encountered him I would do the same thing. I would visibly hide behind a beam and write notes in my notepad. I saw Gene rather frequently during the week because the majority of the time, I left the electric shop by going through the T-G room to one of the boilers and then to the precipitators, where I spent most of my time working at this time in my career (but that is another story for a later time).
Each time, Gene would watch me suspiciously knowing that I was just messing with him, but not exactly sure what I was “up to”. At the time, I wasn’t sure either. So I just wrote down what I saw Gene doing, that way, if he ran over and grabbed my notepad from me, it wouldn’t say anything other than what I saw.
It had happened at the plant a few years earlier when I was a janitor that the company had hired an efficiency expert to monitor the employees at the plant. He would walk around the plant with a stopwatch observing the employees. When he saw them he would write notes on his clipboard. It became very unnerving because you would walk around the corner and there he would be standing writing something down about you.
I went to the Assistant Plant Manager Bill Moler and told him that this creepy guy keeps showing up in the main switchgear by the janitors closet. And every time he sees me, he writes something down. I told Bill that it really bothered me. He explained that he is an efficiency expert and he has a certain path that he takes throughout the day and takes a snapshot of what the workers are doing at that moment and writes it down. By doing that he calculates how efficient we are. It seemed pretty silly to me, because most mechanics when they saw him coming put their tools down and did nothing while he walked by until he was out of sight again.
When I was on the labor crew a few weeks later, and I was blowing coal dust off of all the I-Beams above the bowl mills with a high pressure air hose, I looked down, and through my fogged up goggles I could see this guy standing directly under me. I was about 50 feet above him crawling across an I-Beam with the air hose blowing black dust everywhere. He had crossed my barrier tape to go into the bowl mill area to see how efficient I was being.
This is the type of barrier tape I was using. It is made of woven plastic fibers.
I was so mad I turned off my air hose, climbed down the wall Spiderman-like (no. not head first) and went straight into the A-Foreman’s office and told Marlin McDaniel that the Efficiency Expert had crossed my barrier tape and was standing directly under me as I was blowing down the beams with air. It turned out that the Efficiency Expert had gone upstairs to complain that some guy in the bowl mill had dumped a bunch of coal dust on him while he was monitoring him from below. Evidently, he wasn’t expert enough to know you weren’t supposed to cross someone’s barrier tape without permission, as was indicated on the caution tags that were tied to the barrier tape. From that point on, the efficiency expert (now in lowercase) had to be accompanied by someone from the plant to make sure he wasn’t breaking any safety rules and putting himself at harm.
Long side story short, we turned out to be so efficient, people came from all over the world to study us. Somebody downtown hired the efficiency expert full time, but later he was laid off during the first downsizing. He reminds me of a person on an episode of Star Trek The Next Generation where this guy Lieutenant Commander Remmick comes aboard the Enterprise and he walks around inspecting everything and asking everyone questions that make them uncomfortable and at the end asks Picard if he could come work on the Enterprise. He looked so much like him, that I thought maybe our efficiency expert went to Hollywood to become an actor.
Lieutenant Commander Remmick
Back to Gene Day. I suppose the thought of the efficiency expert may have been going through my mind as I was taking notes about Gene Day at work. Like I said, at the time, I didn’t know what I was going to do with it.
It finally came to me on Thursday morning. This was the last day that Gene was going to be on the day shift, so I figured I might as well do something about it. So when I entered the shop that morning, I sat down at the desk of my foreman Andy Tubbs and began to write. The title at the top of the page was: “The Psychological Profile of Gene Day”.
Using the notes I had taken during the week, I wrote things like the following: “Gene Day walks around the Turbine Generator building with a clipboard in his hand trying desperately to look like he’s doing something important. He constantly hopes that someone is watching him because he dislikes doing so much work to act busy for no reason. At times Gene Day gets paranoid and believes that he sees people spying on him from behind every corner (especially I-Beams). Sometimes Gene Day stands in the middle of the T-G floor staring up into space as if he forgot what he was supposed to be doing.” I’m sure I wrote more, but I don’t remember everything… but I do remember the second from the last sentence. I will save that for a couple of paragraphs from now.
I took my Psychological Profile of Gene Day and went up to the Control Room. I noticed out of the corner of my eye that Gene had just walked over toward the break room behind the Auxiliary Control Panel so I walked over by the Shift Supervisor’s office and I leaned against the top of the large Blue Monitor and placed the Psychological Profile on the top of the monitor in front of me.
Less than a minute later Gene came walking around behind me and seeing the paper on the top of the monitor came up behind me and looked over my shoulder and began to read….. He wasn’t trying to hide the fact that he was reading the paper, and I obviously knew he was there, and the title in large bold letters did say, “The Psychological Profile of Gene Day”. So, he read on.
I heard a few chuckles as he read through my interpretation of what he had done during the week. Then he came to the “Second from the Last Sentence”, as I could hear him reading quietly in my ear… The sentence read, “Gene Day sneaks up behind people and reads their private material over their shoulder!” — Bingo! I had him! When he read that he grabbed me by the throat and started to Throttle me! Shaking me back and forth. This would have been a humdinger of a joke at that point, but I had one more sentence up my sleeve… ur… I mean on the paper….
As I was waivering (is that a word?) back and forth between life and death I managed to eke out something like: “Wait! There’s More!”…. Gene Day let up on me a little and looked down at the page and read the last sentence…… It read….. “Gene Day tries to strangle people who are only trying to help him by creating his Psychological Profile.” That was all it took. Another perfect joke played on Gene Day, and I was able to live to tell about it. When Gene read that he was stunned into dismay. Giggling as hard as he could he retreated shaking his head in defeat.
Now, I know that Gene reads these posts, and he may remember this story a little differently, and I’ll give him that because Gene is older than dirt and his memory isn’t that good. But I was the one that was being strangled, and I still have a vivid image of those few moments. Not only do I, but so do my children, who will one day tell their children, who will say, “Mommy, will you tell us that story again about how Grandpa was strangled by Gene Day that time in the Control Room at the Coal-Fired Power Plant in Oklahoma?”
Runaway Fire Hydrant Leaves Power Plant in the Dark
Don’t believe it when the Electric Company tells you that the reason your town lost electricity for an hour was because a squirrel climbed onto a transformer and shorted it out. The real reason just may be more bizarre than that and the company doesn’t want you to know all the different creative ways that power can be shut off. This is a tale of just one of those ways. So, get out your pencil and paper and take notes.
Power Plant Notepad
One spring day 1993 while sitting at the Precipitator computer for Unit one at the Coal-fired Power Plant in North Central Oklahoma, while I was checking the controls to make sure all the cabinets were operating correctly, suddenly there was a distant boom, and the lights in the control room went out. The computer stayed on because it was connected to an electric panel called the VSP or Vital Services Panel, which in turn was supplied by the UPS system (Uninterruptible Power Supply). That was one of those moments where you may pause for a moment to make sure you aren’t still at home dreaming before you fly into a panic.
The Precipitator cabinets all indicated on the computer that they had just shutdown. I rose from the chair and walked around to the front of the Alarm Panel for Unit one, and found that the lights were only out on Unit 1. The lights were still on for Unit 2. The Control Panel was lit up like a Christmas Tree with Green, Red, Blue and Yellow Lights. The Alarm Printer was spewing out printer at high speed. As the large sheets of paper were pouring out onto the floor, I watched as Pat Quiring and other brave Power Plant Control Room operators were scurrying back and forth turning switch handles, pushing buttons, and checking pressure gauges.
Just this site alone gave me confidence that everything was going to be all right. These Control Room operators were all well trained for emergencies just like this, and each person knew what their job was. No one was panicking. Everyone was concentrating on the task at hand.
Someone told me that we lost Unit 1, and the Auxiliary Power to Unit 1 at the same time. So, Unit 1 was dead in the water. This meant, no fans, no pumps, no lights, no vending machines, no cold water at the water fountain and most importantly, no hot coffee!!! I could hear steam valves on the T-G floor banging open and the loud sound of steam escaping.
I turned quickly to go to the electric shop to see what I could do there in case I was needed. I bolted out the door and down the six flights of stairs to the Turbine-Generator (T-G) basement. Exiting the stairway, and entering the T-G basement the sound was deafening. I grabbed the earplugs that were dangling around my neck and crammed them into my ears. Steam was pouring out of various pop-off valves. I ducked into the electric shop where across the room Andy Tubbs, one of the electric foreman was pulling large sheets of electric blueprints from the print cabinet and laying them across the work table that doubled as the lunch table.
Andy Tubbs – True Power Plant Electrician
When I asked Andy what happened, I learned that somehow when a crew was flushing out a fire hydrant the water somehow shot up and into the bus work in the Auxiliary Substation (that supplies backup power to the Power Plant) and it shorted out the 189,000 volt substation directly to ground. When that happened it tripped unit 1 and the auxiliary substation at the same time leaving it without power.
An example of part of an Auxiliary Substation
I will explain how a fire hydrant could possibly spray the bus work in a substation in a little while, but first let me tell you what this meant at the moment to not have any power for a Power Plant Boiler and Turbine Generator that has just tripped when it was at full load which was around 515 Megawatts of power at the time.
Normally when a unit trips, the boiler cools down as the large Force Draft (FD) Fans blow air through the boiler while the even larger Induced Draft (ID) fans suck the air from the boiler on the other end and blow the hot air up the smoke stack. This causes the steam in the boiler tubes to condense back into water. Steam valves open on the boiler that allow excessive steam to escape.
When the boiler is running there is a large orange fireball hovering in space in the middle of the boiler. The boiler water is being circulated through the boiler and the Boiler Feed Pump Turbines are pumping steam back and forth between the turbine generator and the boiler reheating the steam until every bit of heat from the boiler that can be safely harnessed is used.
When all this stop suddenly, then it is important that the large fans keep running to cool down the steam, since it is no longer loosing energy in the generator as it was when it was busy supplying electricity to 1/2 million people in Oklahoma City. The power is fed to the fans from the Auxiliary substation located right outside the Main Switchgear where all the breakers reside that supply the power to the fans. Unfortunately, in this case, the Auxiliary substation was shutdown as well, leaving the boiler without any fans.
Without fans for cooling, and pumps to circulate the water, the walls of the boiler began heating up to dangerous temperatures. Steam was whistling out of pop off valves, but if the steam drum on the top of the boiler were to run dry, then the entire boiler structure could be compromised and begin melting down. — So, this was serious. Something had to be done right away. It wouldn’t be as bad as the China Syndrome since we were burning coal, but it would have caused a lot of damage nonetheless.
From the movie “The China Syndrome” where a similar emergency existed only in the movie, it was a Nuclear Plant
I have a side story about this picture, but I think I’ll save it for another post because I don’t want to digress from the main story at this point (Ok. Let me just say “Jack Maloy and Merl Wright” for those who can’t wait).
With the prospect that the boiler might melt to the ground in a pile of rubble, it would seem that the main priority was to turn the Auxiliary Substation back on so the fans could be turned back on and prevent the boiler from collapsing. So, we walked out to the substation and looked at the switches that would have to be operated in order to first power up the main bus and then to close to supply power to the two big transformers and the six smaller transformers that supplied the Unit 1 Main Switchgear.
While inspecting the switches where the electricity had gone to ground we found that one of the main insulators was cracked.
A High Voltage Insulator like this
Since this insulator was cracked, we didn’t really want to operate the switch to test if another 189,000 volts would go straight to ground again, especially since one of us would be standing right underneath it cranking the switch. So, we went back to the shop to find an alternative.
By this time the Plant Manager, Ron Kilman arrived in the shop, and understanding the urgency to find a solution asked us what were the alternatives. He was relying on our expertise to make the decision.
Ron Kilman is the one on the left in the plaid shirt
The next solution would be to cut the power over from Unit 2 which was still humming away pushing electricity to Oklahoma City out of the 345,000 volt substation. The cut over would be very simple because the switchgear was designed with this in mind. We analyzed the power rating on the auxiliary transformers on Unit 2 and thought that we might be cutting it close to have them running both sets of fans at the same time, especially since the full load amps of a huge fan starting up was about 10 times the normal rate.
The transformer was rated to handle the load, but consider this. What if this caused Unit 2 to trip as well. With the Auxiliary substation offline, if Unit 2 tripped, we would be in twice the amount of trouble we were currently in. What a day it would have been if that had happened and two 250 foot boilers had come crashing to the ground in a pile of rubble. After reading the power ratings on the auxiliary transformers I was thinking, “Yeah, let’s do it! These transformers can handle it.” Andy was not so eager.
So, we were left with one alternative. That was to shut the switch in the Auxiliary substation that had the cracked insulator and take our chances that it wasn’t going to short to ground and blow up over our heads. I think I was eager to close the switch for Andy, but if I remember correctly, he didn’t want me to be the one to suffer the consequences and decided to close the switch himself. Needless to say. Andy closed the switch, and nothing blew up.
High Voltage Switch similar to the one we were closing
As soon as the power was restored to the switchgear, the fans were powered up and the temperature in the boiler was quickly reduced. The coffee pot in the Electric Shop began heating the coffee again. The power plant was saved from a major catastrophe. That was delayed for another day… of which I will talk about later.
So, how exactly does a fire hydrant shoot water up into the bus work of a substation like the picture of the switch directly above? The culprit fire hydrant wasn’t in the substation, it sat alongside it outside the fence a good 50 feet from the high voltage switch. No hose was attached to the fire hydrant. It was only being flushed out as part of a yearly activity to go around and make sure the fire hydrants are all operating correctly.
Here is the story about how the squirrel climbed into the transformer this time….
George Alley, Dale Mitchell and Mickey Postman were going around to the 30,000 fire hydrants on the plant ground (ok. maybe not that many, but we did have a lot of them), and they were opening up the valves and flushing them out. That means, they were letting them run for a while to clear them out from any contaminates that may have built up over the year of not being used.
Throughout their adventure they had opened a multitude of Hydrants situated out in the fields along the long belt conveyor from the coalyard and around the two one-million gallon #2 Diesel tanks.
Large Oil Tank about the size of the two that are at the Power Plant
The brave Power Plant Men, learned that when opening a fire hydrant wide open in the middle of field had unintended consequences. It tended to wash out the ground in front of the flow of the water shooting out of the hydrant. So the team of experts devised a plan to place a board in front of the hydrant when it would be in danger of tearing a hole in the terrain. The board would divert the water into the air where it would fan out and not cause damage to the surrounding area.
This was working fine, and when they arrived at the fire hydrant next to the substation, since the stream from the hydrant was pointing directly into the substation (hmm. a design flaw, I think), they decided to prop the board up against the fence to keep from washing away the gravel in the substation. Well. When a fire hydrant is opened that hasn’t been used for a year, the first flow of water to shoot out is dark brown.
You may think that this is because the water has somehow become dirty over the past year, but that isn’t quite the case. What has happened is that the pipe has been rusting little by little and the water has become saturated with the rust. So, the water shooting out of the hydrant was full of rust (hence the need to flush them out).
Well. Rust is made of metal. Metal is conductive, especially when it is mixed with water. When the water hit the board, it was deflected into the air and happened to direct itself directly into the high voltage switch in the substation. This caused a circuit to the the ground which, once it created an arc pumped all the electricity directly into the ground.
Normally when something like this happens it doesn’t trip the Main Power Transformer to a Power Plant.
A Main Power Transformer
This time it did. I know there was a few heads scratching trying to figure it out. I think I figured out what happened a little while later. You see… here is the rest of the story….
Once the unit was back online and the emergency was over, someone finally noticed that the telephone system couldn’t call outside of the plant. Well. I was the main telephone person at the time, so the control room called me and asked me to look into the problem.
I checked the telephone computer and it was up and running just fine. Internal calls could be made. Only any call outside just concluded with a funny humming sound. After checking the circuit in the Logic Room next to the Rolm Telephone Computer I headed for…. guess where….. the Main Switchgear….
A ROLM Phone Computer
In the middle of the main switchgear in the back of the room right next to the Auxiliary Substation beyond the back wall, the outside telephone line came into the plant. The first thing it did was go through a special Telephone Surge Protector.
Telephone Grounding Panel
In this picture above, the silver circular buttons on the left side are really an old style surge protector. whenever there was a power surge, the carbon connection in the surge protector would quickly melt causing the circuit to go straight to ground. Thus protecting the rest of the telephone circuit. So, if some kid in their house decides to connect the 120 volts circuit to the telephone for fun to see what would happen, this circuit would protect the rest of the phone circuits. Keep in mind that this was during the early 1990 when “Surge Protection” still had a “mechanical” element to it.
Anyway, when I arrived at this panel and I checked the surge protector to the main line going out of the plant, guess what I found…. Yep. Shorted to ground. Luckily there were some spares that were not wired to anything in the panel and I was able to swap them out for the ones that had been destroyed. — These were a one time use. Which meant, if they ever had to short to ground, they had to be replaced.
Ok. Fine. After a little while, we were able to call back out of the plant, though there was still some residual noise on the line. It was like this… when you called out of the plant, the person on the other end sounded like they were buried in a grave somewhere and they were trying to talk to someone living just like in an episode of the Twilight Episode where a phone line landed on a grave and the dead person tried to call his long lost love from the past.
Gladys Cooper in the Twilight Zone Episode “Night Call”
I didn’t give it much thought other than that I figured the 189,000 volt arc to ground must have shorted out the telephone line since the phone line ran directly under the auxiliary substation ground grid.
It wasn’t until the next morning when the Southwestern Bell repairman showed up at the plant. I knew him well, since he had been working on our phone lines since before the AT&T breakup in 1984. When I met him in the front of the electric shop, he said that he needed to check our telephone circuits. I told him that I knew that we had a problem because we had a high voltage short to ground yesterday and I found our surge protectors melted away.
He explained to me that not only was our circuit affected, but that every relay house from here to Ponca City was blown out. That’s when I realized that the problem was the reverse of the usual situation. What had happened was that the Ground Grid in the substation and the surrounding area (including the Unit 1 Main Power Transformer) had become hot. What do you do when the ground grid becomes charged?
The Ground Grid is what is supposed to protect you when a surge happens, but what happens when the ground grid itself is the problem? In this case, when the high voltage line about 60 feet from the telephone cable surge protector, arced to ground, it fed a tremendous amount of power back through the ground grid. when equipment detected the surge in voltage, they automatically defaulted their circuits to ground. That’s what the telephone circuit die. That’s what tripped the Main Power Transformer.
When the telephone circuit detected the high voltage surge, it shorted to ground (which was the problem), causing the high voltage to feed directly into the phone line and down the line to the next Southwestern Bell relay switch, which also defaulted to ground, trying to bleed off the surge as it went from relay switch to switch until enough of the power was able to be diverted to ground.
That day sure turned out to be a learning experience. I learned that when all the lights go out in the control room, that it is almost assured that the coffee pot in the electric shop is going to stop working. I also learned that in order to coax the plant manager to the electric shop, a major electrical tragedy is one good way. I learned that when shooting rusty water into the air don’t point it at a high voltage auxiliary substation switch. — I’m sure Mickey Postman learned that lesson too. I also learned that just like in Star Trek… whenever there is a dangerous job to do, the Captain is always the one that wants to do it. Does that make sense? Send a Peon like me in there…
I also learned something else about Power Plant Men…. You see…. People like Dale Mitchell, George Alley and Mickey Postman all are examples of incredibly wonderful Power Plant Men. When they were out there doing their duty and something tragic like this, all the Power Plant Men felt their pain. They knew that they all felt guilty for tripping the unit. It didn’t matter that a million dollars every so many minutes was walking out the door in revenue. The only thing that mattered was that these three men were safe.
Mickey Postman
Since I have left the Power Plant, I have found that the idea that the employee is the greatest asset that a company can possess is not a universal idea. You see, there was never the thought that any of these people should be fired for their mistake. On the contrary. The true Power Plant Men did whatever they could to let them know that they knew exactly how they felt. It could have happened to any of them.
Besides the friendship between Power Plant Men, one of the things I miss most about working at the Power Plant is that the employees are held in high esteem as a real asset to the company. Many could learn from their example.
Resistance in a Coal-Fired Power Plant — Repost
Originally posted April 19, 2013:
Resistance is Futile! You may have heard that before. Especially if you are a Star Trek Fan. If not, then you know that there is always some form of resistance wherever you are.
Captain Picard as Locutus trying to convince you that “Resistance is Futile” Like that is ever going to happen
I learned a lot about resistance when I first joined the electric shop at the coal-fired power plant in North Central Oklahoma in 1984. I was assigned to work with Sonny Kendrick and Bill Rivers on the Precipitator during overhauls and when I wasn’t working on the manhole pumps and there wasn’t any other emergencies going on. Actually, from 1984 on, on, the Precipitator for the next 17 years I continued to work on the precipitator… (if I had only known my fate….).
Not only did I learn a lot about resistance, I also learned about capacitance, reactance, transformers, rectifiers, power supplies, diodes, transistors, op amps, and pots (also known as potentiometers). Bill Rivers was the brains of the outfit. Sonny was the Electric Specialist banished to the Precipitator by Leroy Godfrey (See Singing Along with Sonny Kendrick). Bill thought up the ideas and Sonny went to work to implement them. I just jumped in where I was needed.
The Precipitator is the large box between the boiler and the smokestack (maybe you can see this in the Power plant picture). The purpose of the electrostatic precipitator is to take the smoke (or fly ash) out of the exhaust before it went out of the smokestack. The controls for the Precipitator were all electronic at that time. That meant that there were circuit boards full of resistors, capacitors, transistors, operational amplifiers, diodes and potentiometers. These circuit boards controlled the way the power was distributed throughout the precipitator wires and plates through high powered transformers, and how the rappers and vibrators operated that dropped the collected ash into the hoppers.
Bill had me take an electronics course at the Indian Meridian Vo-tech so I would know the basics. Then he taught me all the shortcuts. I had to be able to look at a resistor and tell right away what the value of resistance it was. Resistors are color-coded and you had to learn what each of the colors represented…
This for instance is a 339 ohm resistor with a 5% tolerance.
I was expected to know this by sight. Bill would test me. There was a mnemonic device that I was taught to remember what each color represented, but it is not appropriate to repeat it, so I won’t. It is enough to say that the colors go like this: Black, Brown, Red, Orange, Yellow, Green, Blue, Violet, Gray, White (I will never forget this my entire life). These represent the numbers zero through 9. Here is a full explanation of how to read a resistor….. just in case you are curious, or you are such a boring person that you really need some material to bring up when you are at a party and don’t know what to say:
This is how the color codes work for resistors works
I found that having just the correct amount of resistance was very important. Too much or too less, and everything stops working.
Isn’t it that way with management also? If the management is too resistant to change, then things come to a halt. If they have too little resistance, they lose control of the situation. Depending on the circuit (or managerial decision) and what you are trying to do, it helps to have a manager that has a variable resistance to meet the needs of each situation. Resistance to change is always a balancing act.
During the first two years I was an electrician, the main control panels that controlled the operation of the precipitators were electronic. We spent a lot of time in the lab troubleshooting electric circuits looking for blown (or bad) parts that needed to be replaced. Then we would solder new components on the circuit boards and then put them back in operation. I learned how to be an electronics junky. I became addicted to fixing electronic circuit boards. It was like a game to me.
Later, the precipitator controls were changed to digital controls. That is, they were more like little computers controlling the precipitator. Instead of a bunch of circuit boards dumbly, but cleverly, doing their job, (how many commas can I use in one sentence?), little brains were added that made decisions and reacted to conditions in a much more dynamic way.
What was interesting was that one day Bill Rivers was describing how technology was going to be in the future. He said that some day, we will be able to sit in the lab and look on a computer and see what all the controls in the precipitator were doing (this was 1984). If something isn’t working right, we could just reach over, type a few keys on the computer and adjust the controls. Drink our sweetened tea (a necessary staple in Oklahoma at the time), and then wait for the next crisis…. Then he would giggle at the look of disbelief on my face.
When he was telling me this, I was thinking in my head…. Well, that would be nice, but this sounds more like a pipe dream to me than reality. What does an older guy with six kids from a tool and die company in Columbia Missouri (where I grew up, by chance) know about the future of anything….. well…. anything…uh… new age…. If that is what you might call it… I found out you just don’t really know when you are sitting in front of a true “visionary” with tremendous insight.
Bill Rivers had this incredible knack for telling the future. In 1984 he was predicting computer controls in the control room where you ran the entire plant from a computer on a desk instead of using the “Big Board”.
I love this picture! It makes me feel at home! This was not our plant, but is a Power Plant control room
He said you would be able to call someone on a phone you kept in your pocket or your watch like Dick Tracey.
Dick Tracy in a control room of a power plant sending commands to the hopper nozzle booster pumps
I don’t know what journals Bill was reading or if he just dreamed all this stuff up in his head, or maybe he was a Star Trek Fan that believed that if you can dream it up you can do it. I do know that he picked up on subtle queues and made great inferences from them that seemed astronomically unlikely. However, I have to admit that he caught me off guard a number of times with predictions that definitely came true.
I will talk about this more in a future post, but for now I will say that we did upgrade the precipitator to where you could sit in the control room and monitor and adjust the precipitator controls (all 84 on each unit), and even each of the rappers (672 rappers) and vibrators (168 vibrators) on the roof of each precipitator. With one key on the computer I could send a plume of ash out of the smokestack that looked like the unit had just tripped, and a moment later, clean it up again. This meant that I could send smoke signals to the Osage Indian tribe 20 miles north up the Arkansas (pronounced “Are Kansas”) river, telling them that the Pow Wow would begin at sunset.
Today, I understand that the “Big Board” at the plant is just a large junction box and the plant is controlled almost (if not) completely by computers sitting on the desk. Before I left the plant in 2001, this was being transitioned slowly to computer controls. I have another story to tell some day about this, and how an operator named Jim Cave, a Power Plant Genius and true Power Plant Man of the highest integrity, was snubbed by upper management for speeding this technology along. — Another example of Power Plant Resistance….
But for now…. back to my electronic days… before I began re-programming the Eeprom chips in the precipitator controls….
An Eeprom Chip used in the precipitator controls which I had a very “personal” relationship with…
Bill Rivers confided with me one day that when the new Instrument and Controls department had been formed from the “Results” department that his dream had been to become a part of this team. It meant the world to him. It was where he believed he belonged. It was one of his major goals in life.
There used to be two electrical specialists in the Power Plant. Sonny Kendrick was not always the only one. The other specialist was chosen to go to the Instrument and Controls shop. Bill Rivers wanted to move there also. He definitely had the experience and the knowledge to be a superb instrument and controls person. But Bill had this one problem.
He loved to joke around. He loved to pull strings and push buttons. I have mentioned in a previous posts that Bill would play a new joke on Sonny Kendrick every single day. As I have unfortunately found out in my own life… this tends to make them…. well….. it tends to make enemies out of those who have a chip on their shoulder. Those people who naturally feel inadequate in their abilities or their position in life. To go one step further…. anyone who feels “unloved”….. these people definitely do not like being joked with. They seem to never forgive you. My greatest regret in life is joking around with these individuals.
So, when it came time to choose who would be a part of the new Instrument and Controls shop, Bill Rivers was turned down. It was explained to him that the reason he was not given the job was because he cut off the leads of a resistor when he replaced them. — I’m not kidding. Bill Rivers had the habit of cutting off the leads of each resistor, transistor, diode or capacitor that he replaced…. this is why Monty Adams turned down his request for joining the “elite” Instrument and Controls shop (as he told Bill to his face).
Someone had told the Instrument and Controls Supervisor Monty Adams that Bill Rivers cut the leads off of transistors and resistors when he replaced them so that you couldn’t test them to see if they were all right. Implying that he didn’t want you to know whether he had replaced the transistor or resistor by mistake.
Bill Rivers took several transistors, cut the leads off of each of them and handed them to me and asked me to test them to see if they was still good or if they were bad. I took out my voltmeter, set it to ohms, and proceeded to test them as Bill Rivers had taught me. I told him…. this transistor is good….. this one is bad….
You see…. there is no way to cut the leads off of a transistor in such a way to make it impossible to tell if a transistor is good or bad…. In reality…. you cut the leads off of a bad transistor so that the person working on the circuit board knows that this is a bad transistor and doesn’t use it again by accident. This was electronics 101.
When Bill told me this story, he literally had tears in his eyes. This was because being part of the Instrument and Controls team was part of his dream. His family and the entire rest of his life was decided the day he was told that he was not going to be a part of a team that he believed was his true lot in life.
I remember his exact words as he sat there in the lab alone and told me this story. He said, “… and Monty didn’t know… He didn’t know that you cut the leads…. that is standard procedure….” In Bill’s giggly way, he was crying out loud as he told me this.
From that point on….I knew that the decisions Bill made in his life were driven by that one decision to exclude him from this team. Unlike many of us that could say to ourselves…. “That is their loss”…. Bill kept this pain in his heart each day…. Every decision from that day further was effected by this event.
I calculated it out one day that I spent 414 hours driving back and forth from Stillwater, Oklahoma to the plant and back each day with Bill Rivers (along with Yvonne Taylor and Rich Litzer and occasionally others that needed a ride), and over that time, I became very close to Bill, even to the point of tutoring his son in Algebra (see post: How Many Power Plant Men Can You Put in a 1982 Honda Civic?).
I say this because I know about the pain that inflicted Bill River by a rash decision based on the hearsay of someone that held a grudge. I know how his entire life was changed and how it ended for Bill Rivers as a power plant employee. I know that every decision by Bill after this date was made in response to this one decision. Anyone who experienced Bill after 1983 knows what I am talking about.
I realized that today my own decisions in life help spell out my future. How some little remark may be misinterpreted, or even properly so. I realize as I write this post that how I accept or reject these events in my life, determines the future of my family. After seeing how every event in Bill’s life after that day at the power company was determined by his experience was to his detriment, I am determined not to let the same thing happened to me…..
That is why I have taken on the philosophy in my life that no matter how my actions are misinterpreted, I am determined to remain true to myself. I know what I mean, and I mean what I say, and I say what I mean, and an Elephant is Faithful 100 %.
From Horton Hatches an Egg by Dr. Seuss
Comment from the Original Post:
-
It’s amazing how many decisions are made based on incorrect / incomplete information (at all levels).
That was an exciting day! Another great story. Thanks for the memories.