At my fathers prompting, I thought I'd give a synopsis of what has been going on here at my company for the last week. A little background on me and my employer: We are a small public company (~35 employees, ~20 million in annual sales). We supply our products to large and small retail stores. The products are made overseas, shipped here to our warehouse, and then distributed around the country as orders come in. Our salesmen use laptop computers to take orders, and then dial in to our system and upload them each day. We also receive orders via EDI and customer calls. Our main system is an IBM AS/400. All of our applications were developed in house over the last 10-15 years. Much of the code is antiquated, and we operate in a batch mode similar to larger mainframe shops.

I started working here in Sept. of 99 as a programmer. I am one of two on staff, and have about 10 years experience in programming. This is all first hand info, as I am one of the ones fixing the programs, but I am not going to get so specific that I can be fired. Still, this should be enough to quiet those who would argue my credibility. I asked about Y2K remediation before I started working here, and found out that all the code had been fixed by my boss (the other programmer on staff), who has worked here for about 18 years.

We got a call from the contractor who custom-wrote the software our sales force uses on Dec. 29th. It seems he just found out that they wouldnt be able to take any orders dated after 12/31/99. We dont have the source for that program and the patch hasnt arrived yet, so all orders taken by the salesmen are dated 12/31. We have to intercept them as they are uploaded and then manually change them. Slow, but still manageable.

On Jan. 3rd, the first day we operat after the big day, things start breaking. My boss used the same basic routine to fix date problems throughout our system. Unfortunately, he didnt quite get it right. The order entry program used by customer service wont allow entry of any orders dated after 12/31/99 because the check for orders older than 30 days fails. This one only takes me about = a day to fix. Since similar code was used in other places, other systems have the same problems. AP checks cant be run. Some of the batch entry programs for relieving inventory wont take anything shipped after 12/31. Some reports slipped through the cracks, and are showing incorrect data.

None of this put us out of business, but has required constant monitoring and fixing of the system. We are a small company, and our step-by-step batch operation has kept this from being critical. Our processing never moved to real-time, so errors can be caught as they are found. We perform step 1, check the results, then step 2, etc., and fix any discrepancies as they come up and re-run jobs as necessary. Most things were fixed (except the sales laptop program) by Friday.

But then we ran the larger batch jobs over the weekend. It seems that the program for purging shipped orders older than 90 days decided to erase all shipments made this month as well. Oops. Fix the program, restore the data, and re-run the job. Everyone who needs to deal with those shipments can read the paper till we are done. We will probably make it through the rest of this week without many new problems, but I am expecting similar results to happen when we run the month-end jobs in a few weeks. And again at year-end.

As a programmer, let me give you my opinion of how this relates to the rest of the business world. First of all, we are not the only company having these types of problems. I have worked for 3 other manufacturing companies as an employee, and also spent a couple of years consulting. The code here is of the same or better quality as other places Ive been. Second, we are an old-style company that really only uses the computer to speed up our manual operation. We dont deal with real-time inventory, work-in-process manufacturing, or just-in-time supply chains. If we did, then we would be hurting. Some of my former co-workers must be spending a lot of time at the office right now. Third, we are not out of the woods yet. Not every program gets run every day. A small number get used daily, more weekly, and even more only once or twice a year. There are more Y2K related problems coming.

The big question is what effect will this all have? Beats me. Since the power stayed up, and the phones didnt drop, and the nukes didnt blow, the stock market is probably the only barometer left to look at. Eventually a companys performance will be reported, and the effects of Y2K will be apparent, even if Y2K isnt acknowledged as the reason. But in an era when a company can consistently lose money and yet have their stock value soar, who knows when we can get a real picture of what is happening. For my part, though, I think Ill hang on to my rice and beans just a little while longer.

Coder

-- Coder (Coder@Work.Now), January 10, 2000

Answers

Good post Coder. Thank you. =)

-- Dee (T1Colt556@aol.com), January 10, 2000.

Thanks Coder for the interesting post.

-- PA Engineer (PA Engineer@longtimelurker.com), January 10, 2000.

The staff programmers in my former multi-national company knew there would be a struggle, mainly because higher-level management goals did not permit effective remediation. All of them left in 1999 as well as the top four IT managers! On January fourth, I heard that all of the daily closings from a heavily used ap were being posted as 01/03/1900. They are also running email software that has a known Y2K bug that can crash a gateway. BITR?

-- Slobby Don (slobbydon@hotmail.com), January 10, 2000.

We are bumping along the iceberg now ...

-- bw (home@puget.sound), January 10, 2000.

Excellent post Coder.

-- snooze button (alarmclock_2000@yahoo.com), January 10, 2000.

Businesses that use their Accounts Receivables and inventory as collateral for revolving lines of credit could have some problems producing reports that lenders use to adjust credit lines.

-- lisa (lisa@work.now), January 10, 2000.

Coder,

I just wanted to follow up on your report. First of all - you sounded ligit, ligit enough that I just made a few phone calls to two older clients of mine that are running mainframe systems in either a manufacturing or medical environment. None of that software is mine, I work only in a PC based environment.

Neither had so much as a hiccup. Which is certainly not the end all to this story, but interesting none the less. I am going to put out some more effort into collecting some info from other people that I know, all across the US, and see what gives. Truthfully I have not bothered because I received NO phone calls, OR email, from anyone I personally knew who had problems or needed help in any way.

But I am curious. I will probably post a new thread with what I find tomorrow.

-- H.H. (dontscrewme_2000@yahoo.com), January 10, 2000.

Thanks for the inside scoop coder...........

-- kevin (innxxs@yahoo.com), January 10, 2000.

Coder - Your post is more disciplined, better defined and specific news then I get from the media. Heartily appreciated. Thanks

-- Guy Daley (guydaley@bwn.net), January 10, 2000.

I wonder what would happen if I wrote my post of this type. Neither myself, my family nor the 5 or 6 friends I've contacted about this stuff have had any y2k related anything happen to them, no power outages, bad bills, computer glitches at work. Nothin. I would probably get a "get a life, troll" response or two, if I started a thread with these simple facts.

And don't count on the stock market to be an indicator of squat. It took a pretty major nosesdive 18 months (or so) ago and we were still growing like crazy and primed to grow some more.

Mr. Coder, good luck with your work troubles. I don't think that they are an indication that you need to become "resource independant" however.

-- Mike (mike@noemail.net), January 10, 2000.

Thanks all, for the nice replies. Specific responses:

Slobby Don: I am sure the company you mentioned is one of many. Some poor slob is probably massaging the 1900 data to keep things running.

H.H.: I'm glad I passed your ligit (legit?) test. I don't doubt that there are many companies that aren't having problems. I'm sure that the age of their software makes a difference. Also, I would suspect that newer, unmodified "canned" applications would fare better than heavily customized ones.

Guy Daley Thanks - that is a really nice compliment. You won't get reports like mine from the media, though, because no one in their right mind would report that their company was having these types of problems.

Mike Like they say, "Plane lands safely" isn't a newsworthy headline. However, most of the things you mentioned fall into either the end user or customer category. Our customers certainly don't know about the problems we are encountering. Neither do our vendors. My power company could be running off of deisel generators for all I know. As long as my service doesn't drastically change, I am oblivious to any problems they might be having. But if the problems accumulate until they can't service me any more, then I notice. I just wanted to point out with my original post that everything behind the corporate curtain isn't running as smoothly as you might think.

As for the stock market - who knows what will happen? Certainly not me, which is why I don't own stocks. But it is the only way I know of to judge a company's performance. When I see Lucent drop 28% (or whatever it was) in a day, I get a pretty good idea that everything isn't kosher.

As for "resource independant", well, I prety much am. At least as much as I can be without taking up farming. I live in the boonies, and can get food, water, and heat if the power grid fails or some other disaster arises. But way up here (Maine), that isn't just a Y2K fad. To me it just makes sense.

-- Coder (Coder@Work.Now), January 10, 2000.

Thanks for the info., Coder,

One way things will come out, albeit slowly, will be any firms that are audited by Accounting Firms. Not sure of the implications regarding what you are describing here, except to say there appears to be quite a few orders being processed in 12/99. IF your company uses a calendar year as their fiscal year, then taxes would appear to be quite a tangled mess, since they would show many more sales in '99 and NONE in 2000.

I would also not be surprised if companies experiencing inventory tracking methods have trouble detecting employee theft. There are still so many ramifications to be considered when a company continues to operate with only a portion of their data systems operating dependably.

The human portion of the equation still remains a huge unknown, IMHO.

Regards,

-- (He Who) Rolls with Punches (JoeZi@aol.com), January 10, 2000.

HH(dontscrewme) Another Polly with little to offer but , squeak ... "It's not happening here ." Where is here ? And , how many times have you been screwed ? Do you like it ? S E.

-- Silver Eagle (SEagle@nest.com), January 10, 2000.

Coder, I live in the PeeCee world and you'd be surprised how little awareness there is in here about larger systems, and especially legacy type systems. Our NT shop had few problems, but the bottom line is the world doesn't run on NT. Not yet anyway.

We need real time data here so your post is very welcomed. It indicates that we have data problems, probably wide spread in large systems, probably fixable in most cases, but still eerily remindful of the fact that we are total hostages to streams of binary code and data. Some of it is sick, mangled, and has the flu. Some of us will suffer some consequences. I guess it's not Armegeddon for the world, but that doesn't mean much to the individuals who are sweating it and getting whacked by it.

It could be that nobody understands what we are dealing with yet but I think your post helps put it in perspective.

-- paul leblanc (bronyaur@gis.net), January 10, 2000.

Yes, please, continue reporting. Bennett told us failures would be happening 'across the street', and for those of us who aren't experiencing difficulties, the word from across the street (you) helps keep the matter in perspective.

-- lisa (lisa@work.now), January 10, 2000.

Coder:

Thanks for your post. I have no doubt you are genuine. The temper and tone of the post indicates clearly you are a professional and not a hacker or speculator.

Your post and your replies confirm what I have been saying since 1.1.2000. For those who need a reminder I've duplicated the post below.

It seems that the overnights on 1.1.2000 (and later) have gone much better than I expected (I'm a software guy so I didn't need any experts to explain them to me). The first payroll I believe has gone without a major problem, however, lets wait until each of the time periods I describe below have gone by before we see just how bad it will be. (I really want to see the a fully cycle of government processing as well) Although overnights seem to be ok, as I say most of the errors will happen when dates are used the most. That will at the end of the first month.

And as I stated in my post I'm curious: Why are *you* still checking this forum?:

I'm a doomer and the only reason I think I'm here is that if the world is dying out there from a thousand cuts, some anonymous or not so anonymous whistle blower will let us know here what's happening in their company, although we haven't seen any thousand cuts reprted here (may be a few dozen from news sources, but none from the "inside" as it were), but I'm willing to wait until Jan 31.

But on thinking this through it doesn't make much sense to hang out here. I'd like to have the last laugh on the pollies (which may still happen I believe), but if the companies keep the world together with bandaides it'll make no difference even if the whistle blowers come here as the perception will still be Yawn2K. After all perception is reality, unfortunatly. If the companies don't keep the world together, we'll find out on CNN pronto as the sensationalist reporters kick in on cue (que?).

Yes embeds are "non-issue" BUT IT will have its real rollover test TONIGHT here's why

----- EXERPTS FROM ABOVE FOLLOW ---------

But with respect to IT/database systems etc. The real test for them will be TONIGHT when the first day of overnight processing happens. You see if you done any programming you know that must of the bugs occur at what are known as boundaries. Now 1/1/2000 is a boundary. It is a boundary when you rollover and it remains a boundary until you use it. WE HAVE NOT FINISHED THE FIRST DAY SO THE BOUNDARY IS STILL LIVE AS IT HAS NOT BEEN CROSSED YET. IT will use it very heavily tonight when they process today's transactions. You yesterday night the processed transactions that were checking dates for work between 30.12.99 and 31.12.99. Today they will be processing dates for work done between 31.12.99 and 01.01.00 and will use new date seriously for the first time.

Boundaries cause errors just prior to the boundary, at the boundary and just after the boundary. That is the rule of programming.

So you can expect the potential for the most errors at the following times (amongst other times) for IT systems:

Begining of first month before boundary. Begining of first payroll (etc.) before boundary. Begining of first week before boundary. Beginign of first day before boundary. After boundary has crossed. Begining of first day,month,week,payroll etc. after boundary. End of First day after boundary. End of First week after boundary. End of First payroll after boundary. End of First month after boundary. End of First quarter after boundary. End of First year after boundary.

The vast majority of potential errors come where the vast majority of the date calculations can occur. These occur in the After periods when we reach the End of xxx time.

So hang on folks the ride in IT BEGINS TONIGHT.

-- Interested Spectator (is@the_ring.side), January 10, 2000.

Coder:

Thanks for your post. It was intelligent and concise. Your survival thus far has been due to not being computer dependent, its a convenience. You have a good year of code repair, and it will get to be routine, no surprises.

Thoughts for the future: willl your customers' systems enable them to ocntinue to pay their bills? And will your vendors be able to continue producing and shippin?

-- John Littmann (LITTMANNJ@Aol.com), January 10, 2000.

Coder,

Good to hear that you are well. I remember you from a chat site early last year. if this is the case, you better go rafting with us this summer, you bum!

Warmest regards,

Southside Ed

-- Southside Ed (Im@home.again), January 10, 2000.

Is hard to tell. With everyone sending their posts in 2 and 3 times at a pop, I can't get a post in edgwise... ;-)

smiley

-- Smiley Joe (tongue@side_of.cheek), January 10, 2000.

Interested Spectator, I'd send mail to Chuck or whoever to snip the extraneous.

-- lisa (lisa@work.now), January 10, 2000.

Lisa:

Got an e-mail address for chuck?

The problem arose from some sort of delay while posting earlier today and I'd get a no-response from server message back. So I'd try a few times, checking before I reposted just to make sure the post didn't acutally happen. Then all of a sudden they all appeared. They must have been caching the posts somewhere.

-- Interested Spectator (is@the_ring.side), January 10, 2000.

Int'd: From main page:

This forum is maintained by Time Bomb 2000 Forum Moderators (y2ktimebomb2000@yahoo.com). The maintainer has a private page for deleting old threads, etc.. You can get a summary of the forum's age and content from the statistics page.

-- lisa (lisa@work.now), January 10, 2000.

"Mike Like they say, "Plane lands safely" isn't a newsworthy headline. However, most of the things you mentioned fall into either the end user or customer category. Our customers certainly don't know about the problems we are encountering. Neither do our vendors. My power company could be running off of deisel generators for all I know. As long as my service doesn't drastically change, I am oblivious to any problems they might be having. But if the problems accumulate until they can't service me any more, then I notice. I just wanted to point out with my original post that everything behind the corporate curtain isn't running as smoothly as you might think."

I don't know anyone working in the country's infrastructure but the folks in family and social circles all work for a living and have contact with a multitude of systems over the course of the day. in addition to experiencing no outages or hassles with credit cards etc.

Since rollover, the market has been fairly even, down then up. just like before. For every Lucent that dropped, there was a big gain or gains somewhere else.

I view these grassroots reports as interesting stories but nothing to get excited over.

-- Mike (mike@noemail.net), January 11, 2000.

No-email Mike: These IT failures are interesting to us tech folks. I'm sure they bore you, but it's rather like the childhood 'ooh, take your bandaid off' kinda thing.

Skip it if you can't deal with it.

-- lisa (lisa@work.now), January 11, 2000.

I am the banking administrator for a moderate sized company. Today my boss pointed out that a section of cheques had been recorded in the ledger with a 1900 date. He was concerned that the bank might refuse them. (Subsequently learned that banks are accepting 1900). Research of cheques cashed and mailed back by the bank showed that the date had been truncated to 00 and could not have caused a problem. As my boss (a semi-GI) knows that I am a GI, we had a good laugh over this one.

-- Just another one (rather@not.net), January 12, 2000.

We are taking on water. But we have plenty of duct tape. We know how to swim. God does not permit humans to perish now, as in the past. We are special. The world is evolving into more sanity, more security, more righteousness, more worth. We are in a new paradigm. Old rules no longer apply. The future is bright. Close your eyes and plug your ears. All is well. Ignore any input to the contrary. All is well. Bill Clinton lives. OJ plays golf. God is pleased. Smile.

-- Joseph Almond (sa2000@webtv.net), January 13, 2000.