Rick Cowles take on the latest NERC report

greenspun.com : LUSENET : TimeBomb 2000 (Y2000) : One Thread

Rick has posted his assesment of the latest NERC report, The Electricity Industry and Y2k-Closing on Deadline. He has used an Adobe format.

I do well to get on the computer, don't do the fancy stuff yet. I would appreciate anyone either posting the text version here or givng us a rundown on what Rick thinks.

Since his forum is password protected, I will not give the URL. I hope others here visit his forum and will bring his message back here for us the share.

-- Linda A. (adahi@muhlon.com), August 09, 1999

Answers

The Electric Industry and Y2k - Closing on Deadline

Ok, for those of you who have been anxiously ;-) awaiting my take on the latest NERC report (and current electric industry Y2k readiness in general), I've placed a report online:

The Electric Industry and Y2k - Closing on Deadline

Please note that the document is in Adobe (.pdf) format, and you need to have the Adobe Reader installed as a plugin to your browser. I would appreciate any and all feedback. This document is intended to be a living document, and I will update it as appropriate based on the comments I receive.

-- Rick Cowles (rcowles@waterw.com), August 09, 1999

Answers

Rick, Wold it be possible for you to post your article here? My old, low-memory electronic brain doesn't have the oomph to do Adobe. Thanks!

-- Ann M. (hismckids@aol.com), August 09, 1999.

Ann - I'll see what I can do about a text only version that I could post here. It's a bit long (7 pages) with graphics, etc - that's why it was done in Adobe format. The biggest thing is finding an available few minutes... ;-)

-- Rick Cowles (rcowles.remove@waterw.com) , August 09, 1999.

RICK

Thanks for your great report....I thought NERC, Koskinen and company seemed a bit too optimistic (I've been following y2k closely since May). I am shocked at your figures that a mere 3% (251 of 7941) of the EEI listed companies are represented by NERC. There should be an EEI report....

-- KLT (KLTEVC@aol.com), August 09, 1999.


Most excellent Rick. Hats off, applause.....

I expect what you outline for the electric utility industry parallels all industries to a significant degree. Actually, the electric utilities are probably much, much better off than most.

Back in the seventies when the economy languished for so very long there was a saying that went; "If your neighbor is out of work it's a recession. If you are out of work it's a depression."

Next year; if your neighbors power goes out, Y2K was bad. If your power goes out then Y2K was real bad.

Combine that with all the other potential failures in other sectors of society and it could get uncomfortable for a lot of us, even if our power does stay on.

I agree 100% our focus needs to get very local from here on out.

-- Steve Kube (stevekube@perigee.net), August 09, 1999.


Thanks for the report, Rick. I appreciate your cutting through the smoke and mirrors and getting down to the information we need. I learned, unfortunately, that my NERC region is MAIN - the author of the confidential file we accessed last week.:( (The one that kept listing certain Y2K readiness info as unknown.)

Another thing that bothers me as I read your report, the MAIN report and the NERC report is the reliance of vendor statements. I have heard from several different sources that vendor statements can be highly unreliable or written in such a way that the vendor can loop-hole their way out of any legal problems. As a matter of fact, a friend of mine - a manager at a multi-million dollar foundry was signing Y2K vendor forms and assuring compliance before he even knew what Y2K was all about. A recent (maybe not so recent anymore) article in either PC Magazine or Computer World stated that companies should not count on the vendor statements being reliable.

Well, thanks again for all your great input and the time you take to keep us informed. I'm going out to buy some more candle oil and a couple of more cords of firewood.

Terri Reid

-- Terri Reid (reid9@midwest.net), August 09, 1999.


Ooooooooh, you nasty man. You just want people to be scared spitless, so they'll spend all their money on whatever you're selling. Just like you did when you wanted them to believe another ice age was coming. And just like you'll want them to worry about the sun going supernova next century. Or something like that.

-- Lane Core Jr. (elcore@sgi.net), August 09, 1999.

Lane, you didn't put a smiley after your post. ;-) Someone is bound to misinterpret

-- Rick Cowles (rcowles.remove@waterw.com) , August 09, 1999.

yesssssss!!!!

a lonely voice of truth and reason crying out in the wilderness.

i, of course, was particularly impressed with the third bullet.

now, if only those with ears to hear...

-- marianne (uranus@nbn.net), August 09, 1999.


As a public service for everyone in the forum, especially for those of you who are Adobe-challenged :), here is the text version of Rick's analysis. Insofar as the graphics go, well, hey, I'm in a hurry :) And it ain't easy to translate them, so I just pretty much left things as they were:

On August 3, 1999 the North American Electric Reliability Council (NERC) issued itslatest, and supposedly last, quarterly Y2k report to the Department of Energy. I'm going to resist the temptation to either praise or criticize the latest report, and indeed, the entire task that NERC has undertaken over the past 14 some-odd months. Certainly, the task has been unprecedented in the history of the electric industry. Unfortunately, the overall results remain mixed and inconclusive.

Regardless of how you read the report, the facts remain that:

 Y2k remediation activities are not complete in the electric utility industry.

 The smaller electric companies, represented by American Public Power Association (APPA) and National Rural Electric Cooperatives Association (NRECA), continue to lag the "bulk power distributors" in Y2k preparation.

 The nuclear power industry response to Y2k, as a whole, has been less than adequate. The response of the U.S. Government regulatory agency tasked with nuclear power safety oversight (the U.S. Nuclear Regulatory Commission) has been nothing short of abysmal over the past 12 months.

 The Y2k status of Independent Power Producers (IPP's) is being largely ignored, despite the considerable contribution that IPP's make to the overall stability of the domestic U.S. electric system.

Over the past 18 months, I've had the pleasure (or frustration, in some cases) of working with quite a few electric companies, industry organizations, federal regulatory agencies, and state regulatory agencies. In many cases, the effort put forth has been outstanding. In more than a few cases, however, the efforts have been little more than a public relations shell game. Here are a few examples of companies I've dealt with that lean more toward the second type:

 A large Southeastern U.S. electric cooperative that is reporting "Y2k readiness" had not really even started a Y2k program as recently as 12 months ago.

 Another that reports "readiness" is a majority owner of a multi- unit nuclear facility that had not started any Y2k work as of the first quarter of 1998.

 Still another is a large independent system operator that didn't award a Y2k program management contract until the fourth quarter 1998.

 And lastly, an electric company serving a major metropolitan area, whose Manager of Engineering Projects couldn't convince executive level management, as of 14 months ago, that Y2k was a big issue and therefore couldn't get commitment of a budget for Y2k inventory and analysis. This same company had not even begun looking at their system operations center during the same timeframe, and I know for a certainty that they had platforms, systems, and hardware that flat out would not make the transition to Y2k.

While there's no question that there are many Y2k-stellar performing electric companies (I've had the good fortune to deal with some of them, too), it is difficult for me to believe that my experience with this sampling of companies is unique and an aberration for an entire industry that says it has been working on the problem since 1995.

Here's my bottom line: while many electric companies took Y2k seriously, and I believe that they believe they've put the requisite effort into dealing with the issue, too many electric companies have not taken the issue seriously. There are companies that have simply gone through the motions, assigned "less than the best" personnel to deal with the issue, and have signed off on completion simply to get the industry associations (and pundits such as myself) off their back.

My sense is that a significant minority of the companies showing up on the NERC reporting list as "ready" or "ready with exceptions" is being a little fast and loose with their Y2k readiness statements. This feeling is based on the following factors:

 My direct involvement on the Y2k issue with many electric companies, industry trade organizations, and state level Public Utility Commissions

 Raw statistical analysis of NERC and other industry reporting

 Historic management reporting practices

Since the day that the latest NERC report was issued, I've been asked many times if I believe the proclamations that the North American electric industry is ready to make the century transition. My short answer is that, no, I do not.

A Deregulation Digression

If I am to believe that the entire electric industry on the North American continent has seriously prepared for Y2k, then I am required to believe that the companies I described above finally committed the necessary personnel and funding resources to tackle this major project. I am also required to accept that these companies have worked the project in good faith from end to end in all business units, in less time than it takes a good sized inter-disciplined team to evaluate the impact of a single major IT system replacement. From a management perspective, I would also have to believe that the executives in every electric company, some of who felt that Y2k was a budget eating non-issue to begin with, finally got religion.

The electric industry, in preparation for deregulation and competition, has changed dramatically in the past ten years. Maintenance budgets have been slashed. As an example, in the world of Transmission and Distribution, fix-on-failure has become the maintenance mode for most equipment, rather than periodic maintenance and/or replacement prior to failure. Personnel downsizing has been endemic - the "history" of most electric companies has walked right out the front door. As shown by two summer's worth of power interruptions and disruptions in the mid-West and mid-Atlantic states (and acknowledged at all levels of industry and government), distribution systems are being strained to the breaking point.

And as a recent expatriate from this environment, am I to believe that Y2k has been addressed properly and with the necessary level of attention in every electric organization from the Bay of Fundy to Coronado Island and all points in-between?

I might as well believe in the tooth fairy.

Pick a Number

The August NERC report is as prominent for its omission of information as much as for the conclusions it draws. I'm only going to latch onto one statistic: industry participation.

While NERC and the electric industry contend that the information gathered represents a vast majority of the generation and transmission assets in North America, it is hard to reconcile the numbers and believe that a mere 3% (251 of 7941) of the EEI listed companies represent the only "critical" segment of the electric power industry (see Table 1).

Table 1 - NERC / EEI Comparisons The first column represents the utility type. The second column represents the number of R or RE respondents to NERC in each type The third column represents the total number of companies in each type (USA) from EEI Utility Type NERC 1 EEI 2 Canadian (CA) 14 (n/a) Coop (CO) 28 875 Govt (GO) 13 69 Investor owned (IOU) 94 222 Muni (MU) 37 1885 Non-utility (NU) 57 4247 Power District (PD) 8 73 Power Marketer (PM) 0 570 Grand Totals 251 7941 Notes: NRECA received 90% response rate from Co-ops APPA received 86.3% response rate from Muni's No EPSA data for Non-utility generators or EWG's No report for Power Marketers Now, based on survey responses from the participants in column 2 of the above table, the electric industry as a whole, represented by NERC, has drawn the conclusion that Y2k will be a non-event for most of us (at least power-wise). But let's take a little closer view of some different numbers for one region of the country - my home state, New Jersey.

1 August 1999 NERC Report to DOE, Appendix B 2 http://www.eei.org/issues/comp%5Freg/3electri.htm

According to USDOE, Energy Information Agency statistics 3 , nuclear power provides somewhere around 30 percent of generation capacity required in the state. IPP's (non-regulated power generators) provide another 50 percent 4 . None of the nuclear plants in NJ are yet Y2k ready 5 . No one knows about the IPP's (including NERC, who I have queried on the subject repeatedly). Is New Jersey unique? Yes. But so is every other region in North America; that's my point. And that's why I think it's absolutely ludicrous to make global pronouncements about overall electric industry Y2k readiness. We simply don't know.

JFK, UFO's and Y2k (Apologies to Richard Belzer)

Do I think there's a "conspiracy of silence" to keep the truth from the public? No. Do I think that any of the reporting companies are flat out lying? No. What I think is that some of the major players are, in fact, being fed superficial information from some of their reporting companies who don't take this thing seriously. And so, it comes down again to dilution and distillation of the data stream as that data is being fed up the "chain of command". When the data is analyzed at its final destination point (NERC), it is further homogenized and presented as a picture of the entire industry. I fail to see how anyone in the industry (or outside the industry, for that matter) can draw specific, supportable conclusions from the information provided that apply to every geographic region in North America.

In the final analysis, we are only marginally closer to being able to draw Y2k impact conclusions from all of the industry and government reports. I think we can reasonably expect that, barring an unforeseen cascade type event, the power will remain on for a majority of the North American continent. The question then becomes: What percentage of this vast continent remains at risk for power failure? I can't say, and neither can anyone else, because no regional risk studies have been or will be conducted prior to the actual event. And above all, from an infrastructure standpoint, Y2k has been and will remain a regional and local event. If you're in one of those unlucky regional or local areas, then it will not matter to you whether or not the "majority of the North American continent" is unaffected. If the North American electric industry makes it through Y2k mostly unscathed, it's because the industry as a whole got lucky this time, not because of a dedicated, across the board effort that every single one of the 7,941 individual participants subscribed to. And if one large power company (or a few smaller ones) take a hit on 1/1/2000 or the days following, the entire industry loses. It loses the confidence of a public that has become dependent on an increasingly degraded and brittle electric power infrastructure.

3 http://www.eia.doe.gov/cneaf/electricity/st_profiles/new_jersey/nj.htm l 4 Most recent statistics, as of December 1998: http://www.eia.doe.gov/cneaf/electricity/ipp/t24c.txt 5 http://www.nrc.gov

So Where Do We Go From Here?

In the time remaining prior to 01/01/2000, it is this writers opinion that Y2k readiness assessments must be more regionally focused. These regional assessments, conducted along the boundaries of the NERC regional council areas (depicted in Figure 1, below), could provide a greater level of granularity and local confidence rather than an overall proclamation that the entire continent is Y2k ready.

These assessments must be risk based rather than operationally based. In very simple terms, if the chances of losing power in one of these regions today are one in a thousand, what are the chances of losing power to any local jurisdiction within the region on 1/1/2000 (or the weeks thereafter) for reasons due to or exacerbated by Y2k issues?

Figure 1  NERC Regional Councils

Why is it important that this risk analysis be regionally focused? For the reasons provided earlier in my description of the electric power infrastructure in the state of New Jersey. Each state and region has a unique mix of power generation and transmission assets. Each region experiences widely different climate conditions during December and January  for example, the risk of a severe weather related event concurrent with Y2k is much greater in upstate New York than in Florida.

Only by understanding the local level of potential Y2k risk can businesses and consumers make educated decisions regarding their own preparations for Y2k. At this point, we have only been given broad, continental assurances; we have not been provided with local risk parameters. To engender the confidence of the public that Y2k is truly going to be a non-event for the electric industry, it is critical that the industry takes its risk analysis down to the local level. After all, short of comets crashing to earth or a global nuclear confrontation, everything that affects our daily lives is locally based.

The above recommendation being made, I do not expect that either the electric industry or any government agency will conduct such a risk analysis in the short time remaining between now and 1/1/2000. So, it is incumbent on every business, industry, and individual to conduct a personal risk assessment. A good starting point is to use the document, Beyond Code and Controls - Y2K Risk Analysis in the Electric Industry, available at the euy2k.com website 6 .

It is not the time, contrary to what both NERC and the Presidents Y2K Council seem to suggest, for the electric industry to declare victory over Y2k. Government and industry must continue a diligent effort to minimize Y2k impacts, and not become complacent or rest on perceived laurels. The health, safety, and security of each one of us depend on it.

6 http://www.euy2k.com/riskmgmt.htm

-- Drew Parkhill/CBN News (y2k@cbn.org), August 09, 1999.


Rick, Could you justify your statement that the US grid is "brittle"? There has only been 1 major blow to the grid in more than a decade. Other nations have a truly brittle grid where power is off 4 hours each day. Localized outages will always be with us, even after many people and businesses own their own fuel cells and solar panels.

Could you really testify to a Senate panel that the US grid is brittle?

-- E. Dames (halebopp@comet.com), August 09, 1999.


Ask the residents of Chicago (where the lights were out for up to three days and somewhere around a hundred people died a few weeks ago).

Ask the residents of Southern New Jersey, where a few weeks back the power was out for a few days because so many transformers and substations failed during the recent East Coast heat wave.

Read one or more of the following articles:

Science Daily

San Francisco Chronicle

NRC SECY 97-246 - Grid Stability Issues

When you're done, come on back and we can talk about it. Oh, and you might want to check out some of the studies and documents on NERC's website (www.nerc.com) and the DOE Energy Information Agency (www.eia.doe.gov).

-- Rick Cowles (rcowles.remove@waterw.com) , August 09, 1999.


Ah, jeez, E., I almost forgot:

The San Francisco outage last December where a lousy job of grounding in a substation took out the entire city for a day.

Ok - this is way OT - so it's the last post I make to this particular question.

-- Rick Cowles (rcowles.remove@waterw.com) , August 09, 1999.



-- Critt Jarvis (critt@critt.com), August 09, 1999.

Thousands were without power again today in Chicago, I have not yet been able to find a link to this info. I don't know the cause, an underground fire was mentioned.

Thanks to Rick for your work, it is appreciated.

What was that MAIN doc.? I too am in the MAIN region, and am just a bit curious. ;-)

-- Deborah (infowars@yahoo.com), August 09, 1999.


Actually, I take that back, I did find a link. It's to a local tv station, so, I'll post the text as well, because I don't think this will stay up too long.

http://www.cltv.com/

More power outages, but not because of heat

Monday has been another day of power outages for Com Ed customers on Chicago's North Side. No hot weather to blame this time.

Instead, Com Ed says an underground fire near Cortland and Southport destroyed several power cables.

The fire was reported about 7:45 Monday morning -- disrupting service to eight thousand customers. About five thousand were still without service by mid-afternoon. It will be into the evening before everything is repaired.

Com Ed spokesman Keith Bromery said, "There are some customers who will be out until around 10. We're proactively calling them and telling them what the estimated restoration time is."

The outage also affected area stoplights as well as traffic on the Kennedy -- with police shutting down the North Avenue entrance ramps for several hours.

-- Deborah (infowars@yahoo.com), August 09, 1999.


Let's see, only one major hit in the last ten years. would that be the 7 or 10 state Western US outage last fall, or ......

Chuck

-- Chuck, a night driver (rienzoo@en.com), August 10, 1999.


BTW, the reason cascades happen is someone doesn't pull the plug fast enough to protect their equipment, and they get caught by the grid failure. the April test tested communications capability, but the "manual" capcbility tested, adds prescious (sp) seconds (actually a minute or so) to the response time. Given this, I wonder that Rick figures that a cascade won't happen. 'Course, I could be all wet, but the above was given as the explanation for the spread of the 2 or 3 East Coast Outages (circa 67, 78(??), etc.).

Chuck

-- Chuck, a night driver (rienzoo@en.com), August 10, 1999.



This is the info re: MAIN

Go to:

http://www.maininc.org/committees/oc/725/Y2KJuly15.pdf

This report is for the MAIN region's own systems, not for individual electric utilities. The many details should satisfy anyone. I can't believe they left this as a clickable .pdf file on their public domain list of documents, but unfortunately I'm not so nice that I won't take advantage of it.

-- Bonnie Camp (bonniec@mail.odysssey.net), August 02, 1999

-- Terri Reid (reid9@midwest.net), August 10, 1999.


Thanks Terri!

-- Deborah (infowars@yahoo.com), August 10, 1999.

Moderation questions? read the FAQ