It seems there is a developing problem with gas lines,chemical processing facilities etc.,is this the direct result of the buffers filling up?This does appear to be escalating.If they are doing this and thereby only now starting to appear more frequently,how long will it be for their peak period,if known???Thank you. I want to thank those responsible for maintaining this forum.It may be one of the few sources available to people looking for a comprehensive source of fast and pertinent information.Thank you.

-- J (jax@borg.com), January 14, 2000

Answers

Okay, Andy, explain it to him...

-- Julie (teachers@favorite.com), January 14, 2000.

JULES, let me think about this for a nanosecond...

-- Andy (2000EOD@prodigy.net), January 14, 2000.

Uh, OK, we need to ask my mate Shuggy - he is ***THE EXPERT***

-- Andy (2000EOD@prodigy.net), January 14, 2000.

Oh! Drat! Where are Flint and Cherri when you need them? I am definatly NOT an embedded programmer, last ROM I burnt was in 1989 but... some of the threads posted earlier indicated that software that uses embeddeds might be affected as much as a month after the embeddeds rolled over. This is not a "buffer filling up" so much as it is an embedded bit of software (a ROMed module -- something on a chip and therefore not easy to change) generating bad data which a true bit of software (say an accounting package or a regulating system of some sort) can't handle. I don't really know what the term, "buffer filling up" means in this context.

-- Michael Erskine (Osiris@urbanna.net), January 14, 2000.

Look at the computers that displayed 2000 as either 19100 or 1910. When those machines computed the year, the century field stayed "19" while the years field went from "99" to "100". Buffer overflow came into play when some date fields couldn't display "100" and cu it off to read "10".

Now imagine there's an embedded device somewhere that's date stamping its operation and it suddenly went from storing opeations as 1999xxxx to using 19100xxx. That example of buffer overflow would limit the device from being able to date stamp 9999 operations per year to handling 999 operations per year.

Or imagine if the device was designed to shutdown a production machine at 1000 operations for a safety inspection. It would never reach that limit and could keep the machine running with growing safety problems. And someone could get hurt or killed.

As far as the peak period for these things, it depends on how the particular device is used and programmed. Some devices might have an overflow situation on the first full production day of the year. Some might not for years, if they track somethig that only occurs a few times per year.

The second situation is what is troubling when considering safety devices affected by Y2K. These things might already be disabled by Y2K problems, but the failure of those safety devices won't be found out until one fails to work when it's needed.

WW

-- Wildweasel (vtmldm@epix.net), January 14, 2000.

Perhaps this will help you... it is a segment from a recent thread by Dr. Paula Gordon in which she submits a statement from a top embedded systems expert.

This engineer writes:

"... First embedded systems do not have a standardization program in place. In essence there can be several ways programs are written. Now to the Y2K question. There are buffers in most embedded system programs. This buffers can be varying in size. When a command is registered with an embedded system and it is improper or not accepted it can place the command in the buffer. It could be doing this by the hour, day, week etc...

Now if I have a program that does not recognize year 2000 it will search for the date (Year) for a period of time. If it does not find it then a loop is created and placed in the buffer. Now comes the problem. When the buffer is full the system can shutdown, be degraded or begin to act up. The concern is that it could take hours, days, even weeks before the buffer is full.

Simple Example: A fire alarm panel each day at midnight registers the date in this format- Month Day Year- Now on Dec 31, 1999 at midnight it does not recognize year 2000. So it attempts to complete the command "store date".

When this fails over a period of time a loop is created in the buffer. This continues for two weeks. At the end of two weeks the buffer is full. No place to send the command so the system shutdown, becomes degraded, or begins to send out erroneous commands.

Concerns

Shutdown the system and restarting could clear the buffer and then the process restarts again.

Degraded systems could fail when needed the most.

Programs that are running on erroneous commands are sent out, i.e., open valve to 30 percent rather than 10 percent at the temperature of 70 degrees.

Hope this helps explains things that can go bump in the night."

[End of quoted material]

These comments help explain why some embedded systems failures may not become apparent for over a week or more after a trigger date or restart.

I would be pleased to pass e-mail along to the engineer who offered these observations if anyone wishes to get in touch with him. (He did not wish his name and contact information posted.)

I have shared the views of this engineer in recent days with officials at the National Institute of Standards and Technology (NIST), the General Accounting Office (GAO), and the President's Council on Year 2000 Conversion, among others. Officials at NIST and GAO have indicated that they would be getting in touch with the engineer.

-- RC (racambab@mailcity.com), January 15, 2000.

Oh,

I almost forgot... just FWIW... here's an appropriate segment from the article on TAVA's remediation of one major oil co.

This snippet explains the 30 day problem...

http://www.prepare4y2k.com/embed1.htm

"equipment successfully made the January 1, 2000 transition and was allowed to continue. Just over a month later, when checked again, the date on the equipment was January 34!"

These sorts of problems can either shut down a system or let it go on rolling with no documentation of how much product has been delivered or when... to wit... (citing again the TAVA report)

"Some of the more interesting findings were that the catalytic cracker would fail and the refinery could no longer make gasoline. One of the more troublesome findings was that the analyzers would continue to work but would send erroneous data. The proprietary networks from the control systems to the analyzers would fail. The inventory and analysis would take 7 weeks and cost $122,000. The conversion for two units would take an estimated 15 weeks and cost $760,000." "In an overview of all these facilities, problems were found in the sensors & analyzers, lab equipment, programmable Control Systems, embedded systems, SCADA systems (Supervisory Control And Data Acquisition), distributed control systems, human-to-machine- interfaces (HMI's), networks, computers, third party applications, and Operating Systems."

"This points out the need for continued testing, especially at the system level."

Hopeful "J" this will help answer your question.

-- RC (racambab@mailcity.com), January 15, 2000.

I would like to thank all of you for this information.it is appreciated.I do get the idea that there is a vast wealth of knowledge out there which does humble me.I also realize that there will be some problems.How severe and their affects we will have to wait to see.THANK YOU ALL.

-- J (jax@borg.com), January 15, 2000.

Mostly the "buffer filling up" mumbo jumbo is an effort to save face on the part of those whose predictions at rollover didn't pan out.

No doubt computer glitches will occur (they certainly have on either side of Jan 1); the "buffer" notion is so loosely defined that it enables ALL subsequent glitches to be attributed to Y2K until eventually a catastrophic malfunction can be blamed on Y2K regardless of when it happens.

-- I'mSo (happy@prepped.com), January 15, 2000.

ImSo,

It doesn't really matter what failures are blamed on -- Y2K or some unknown glitch. What matters is whether the failures occur, and at what rate.

One gas line explosion -- a glitch. A hundred gas line explosions in a week -- a cause for anxiety.

If gasoline production is choked off because of problems in refineries, it doesn't make a damned bit of difference what the real cause is -- the effect is that you aren't able to fill your car up, or that your are paying some inflated price.

And, since you've challenged the credability of the buffer overflow problem, please explain to us the "REAL" mechanism involved. We await your enlightenment.

-- (4@5.6), January 15, 2000.

The real mechanism: Embedded chip buffers do not "fill up" or "overflow" because a wrong date is returned.

You can:

1. Accept this or

2. Keep waiting

-- I'mSo (happy@prepped.com), January 15, 2000.

The whole y2k issue from day one has been extremely polarized. For every statement made by someone, another person offers the exact opposite opinion. For non-IT folks like me, sorting out the truth is difficult at best.

Therefore "I'mSo", I choose your option of Keep Waiting, as no one has convinced me either way.

I really dont think you answered 4@5.6's: "please explain to us the "REAL" mechanism involved.

What I think this person wanted (or at least I do) to hear is what is the real mechanism causing this delayed embedded systems problem. Or are the reported failures not associated with embedded systems. Or is it that nothing is failing after the rollover? Which is it? I am not trying to be a smart ass here - I honestly do not know what is causing the reported failures occuring well after Jan 1. Please offer your expert opinion.

-- ilander -- (ilander@minot.com), January 15, 2000.

"What I think this person wanted (or at least I do) to hear is what is the real mechanism causing this delayed embedded systems problem. Or are the reported failures not associated with embedded systems. Or is it that nothing is failing after the rollover? Which is it?"

Ilander,

You are asking for a summary of this board so far. Your question presupposes that you need an "expert", and the problem, as you stated it, is that experts will happily come from both sides, especially now that rollover has occurred. Let me suggest that you can apply critical thinking independent of technical expertise, and come to the conclusion that the chip buffer problem is overblown. Let me further suggest that any given programmer/chip designer/system engineer/etc etc sees such a small part of the overall picture that his/her expertise isn't very predictive; indeed this is the very thing that is frustrating those who were convinced, based on their own expertise, that Y2K was (and is) going to be such a large issue. They STILL cannot accept that it wasn't and isn't much of a problem.

Would you consider attacking the problem as follows instead of learning chip design and then seeing if you agree with 'experts'?

1. Ask yourself how many systems that are date aware need to look into the future. I think a reasonable answer is "a lot" since any programming that fetches the date generally needs to look forward and backward--that is, the calendar date is used frequently.

2. Ask yourself how many systems failed prior to rollover. I think a reasonable answer is "a lot" since computer glitches seem to abound. For the sake of argument, assume they are ALL date-related embedded chip failures. In reality, of course, only some fraction are date-related, of which only a fraction are embedded chip failures, of which only a fraction are serious, of which only a fraction are critical.

3. Ask yourself what has happened so far to the world's ability to function secondary to computer failures. I think a reasonable answer is that a lot of systems have failed for whatever reason without much macro effect; certainly MOST large companies had some significant computer problems in the past year. Yet at a macro level, an external observer perceives only the usual ebb and flow of human experience. For the most part Y2K is lost in the noise of our everday lives.

4. Now ask yourself this: Do you care about emebedded chips? Do you want to go on parsing arguments for "buffer overflow" when computer systems have been failing all over the place for YEARS without disrupting society? The experts now hanging their hats on delayed embedded chip buffers overflowing are from the same pool who, two or three years ago, predicted that major disruptions would start in 99 and continue. At a macro level, IMHO their opinions have turned out to be gross overstatements so far, and there seems to be no reason to suddenly give them credibility now.

I know this has dodged the entire technical issue, and I know it is too brief, but I hope you get the gist. Let me give you an analogy:

I am not a mathematician. I would venture to guess a flat-earth theorist well-versed in math could give me arguments the earth is flat and its roundness is an illusion. Those arguments would persuade some, and I couldn't counter them mathematically. But because I get to go outside, and fly in planes, and don't believe there is a conspiracy to present the earth as round, I know in my heart the reason this guy thinks the earth is flat is that it is part of his belief system, and not his mathematical expertise.

Not the best example, but...

-- ImSo (happy@prepped.com), January 15, 2000.

-- I'mSo:

Thanks for your answer. Your answer, along with an answer someone else emailed to me, is in the process of convincing me that the embedded system "problem" may not be that big of a problem afer all.

I was first convinced it was a problem after reading Mark Fraustchi's site. My belief that embedded's posed both a hidden and delayed threat was reenforced after reading Paula Gordon's site. It is my recent understanding that Mark nor Paula have any actual embedded experience.

It is easy for me to see how someone studying a given subject can wind up with an erroneous opinion if they have no practical hands-on experience in that subject. Instead of experience being the teacher, the student winds up picking and choosing what is learned and will likely form a very unreal opinion. Unfortunately, this unreal opinion is then passed on to others.

Oh well, live and learn.

-- ilander (ilander@minot.com), January 15, 2000.

ImSo; Interesting that you should pick the "flat earth analogy", now if you will kindly show me a scientist of any cloth, let alone a mathmatician, who will vouch for the earth being flat. I will accept your analogy, until then I (as a degreed Computer Scientist) will apply "critical thinking".

-- Michael Erskine (Osiris@urbanna.net), January 16, 2000.

Michael, you couldn't have said it any better.

Furthermore, if embedded systems were so Y2K benign, just why did so many companies, world-wide, shutdown their manufacturing processes for rollover? Start-up costs are enormous you know, with tons of wasted and out-of-spec production for several days thereinafter. This joke alone had an eartag close to 0.5% of world GDP. So was it that, worldwide, companies weren't as sure about embedded system performance as Imso (sp?) is trying to put it?

And what about Y2K bunkers, worldwide, governments included?

No one really knows yet (and possibly never will) how embeddeds may react under enormously varying circumstances.

Take care

-- George (jvilches@sminter.com.ar), January 16, 2000.