Just yards from the throngs of commuters who rush through the Lindbergh rail station each day, MARTA stashes five years of mistakes that cost millions of dollars.

Past two security doors, in a gray concrete building, a 20-by-20 room holds nearly 500 pieces of computer hardware the transit agency never expects to use. The equipment includes stacks of high-end modems, circuit boards and network routers, as well as a $1.26 million IBM mainframe mothballed three years after it came online.

Many of the items remain in their original shipping containers, and most were never installed. Despite that, MARTA spent tens of thousands of dollars on maintenance contracts to make sure some of the unused equipment would run properly.

MARTA outfitted the storage room in 1999 to be a state-of-the-art computer center. But after spending $299,000 to equip and secure the building for computer operations, MARTA laid the plan aside, making the space perhaps Atlanta's most well-appointed storage closet.

Purchasing documents show the transit agency spent $2.4 million to renovate the space and to buy and maintain about 30 of the big-ticket items stored there. Officials say they don't know the total cost of the other items in the room.

The money would have come in handy in recent years as MARTA has eliminated more than 600 jobs and cut back bus and rail service to balance its books. The transit agency continues to face budget shortfalls and is looking for more than $1 billion over the next decade to expand train and bus service.

Officials concede MARTA can recoup only a fraction of its losses on the equipment -- perhaps only pennies on the dollar.

"Most if not all of it will probably be sold as surplus," said Shyam Dunna, MARTA's chief technology officer.

General Manager Nathaniel P. Ford said new controls would prevent future errors in technology buying, but he does not intend to dig deeper to see who made purchasing mistakes in the past. "My frustration is that I'm spending time looking back and I've got to get this place rolling," he said.

MARTA officials blame some of those mistakes on continually shifting priorities and leadership in its Department of Information Technology, known in-house as IT. Dunna in April became the department's seventh manager in seven years.

"It's been one of the troublesome units in the organization," said Ken Gregor, MARTA's board chairman and until 1994 its general manager. "There didn't seem to be a lot of controls over there."

Tighter controls are crucial because IT will be the backbone of several projects that MARTA officials hope will lure back riders who have abandoned the transit system. Those include new fare gates and fare boxes that will cost $182 million and a $24 million system to track buses and keep them running on time.

Ford says he's "extremely confident" that tighter oversight will end years of IT mismanagement. But he concedes the repeated discoveries of technology snafus have shaken managers' confidence.

"It has affected our entire authority," Ford said, "in terms of 'Well, what else is out there that we may not have made a good decision on?' "

Unexplained expenses

MARTA's current technology managers say they can't explain some of the department's costliest expenditures.

In late 1998, records show, the transit agency spent $394,121 to upgrade its mainframe computer, even though officials had already ordered a new one to replace it. Then, in October 1999, MARTA stopped using the older computer, even though the upgrade had included a Y2K patch to ensure it would still function after Jan. 1, 2000.

Officials acknowledge MARTA continued to pay IBM for maintenance on the mainframe for more than two years after unplugging it. A 1998 invoice from IBM showed the maintenance cost $27,356 a year. MARTA discontinued the service in mid-2002.

Files provided by MARTA contain no explanation for the agency's decision to stop using the mainframe.

Darold Hamlin, MARTA's technology manager from 1997 to 1999, said the agency upgraded the mainframe so it could be used to store backup data. As such, he said, it would have been the central component of a project known as the Disaster Recovery Data Center once planned for the renovated Lindbergh space.

But MARTA never completed the disaster recovery center. Hamlin said a plan to use the computer space at the Lindbergh station was scrapped because it was deemed too costly. His successor as MARTA's technology chief, Degas A. Wright, wrote in a June 2000 memo that he decided moving computer operations there would "not be in the best interest of the authority." Wright declined to discuss his reasoning during a recent telephone interview.

Lindbergh was not the best choice for the disaster recovery center, Hamlin said, because it was on the same power line as the data center it would back up. The agency had built a space near its Candler Park rail station for use as a data recovery center, he said. But the agency changed its plans when MARTA police took the space for use as a communications room, he said.

An internal audit in June 2000 criticized MARTA's failure to use either space as a computer room after paying for the special renovations. A draft of the audit said MARTA could expect criticism "for not effectively utilizing public funds" if the rooms were not put to their intended use.

Auditors deleted the language from their final report after objections by Wright, who told them that MARTA was putting the Lindbergh space to good use as a secure storage area for new equipment.

Unpacked equipment

In addition to the mainframe, MARTA spent tens of thousands of dollars maintaining other equipment that was never unpacked, records show.

By 2001, for instance, the agency had purchased 36 diagnostic units called "sniffers" -- black boxes about the size of a VCR and costing $10,000 or more apiece -- that ferret out computer network problems. Only about seven were installed, Dunna said.

For 2002 alone, MARTA paid $109,711 for maintenance on all the sniffers even though most were sitting in storage. Records show the maintenance service was discontinued in December, when MARTA balked at paying the $126,233 cost of renewing it for another year.

Files provided by MARTA did not show the agency's costs for sniffer maintenance for other years, nor do they explain why that many sniffers were purchased.

"I don't know why they haven't been installed," said Dunna, who joined MARTA last June. By now, he noted, "the equipment is already getting obsolete."

Transit agencies typically don't stockpile technology items, both because of the expense and the risk that they will become outdated, said Donald McCanless, technology chief at Washington's transit agency.

"We keep a very limited supply of spare parts," McCanless said. "We buy them right before we need them."

MARTA auditors in 1999 and 2001 criticized the IT department for losing track of maintenance contracts. Auditors said the agency had paid $299,390 for two years of maintenance on software it wasn't using while allowing other critical maintenance agreements to expire.

MARTA says it will spend $1.8 million this year on IT maintenance, down from about $3 million in 2001.

MARTA officials blame the carelessness, at least in part, on the constant shifts in leadership. "When you have a huge turnover in an organization, some folks moving in this direction, they go and another person comes in, and they move in that direction," said Gregor, MARTA's board chairman.

Audit could expand

Possibly as early as this week, MARTA expects to release an audit of three years' worth of telephone and cellphone spending.

But one former MARTA official contends the agency will never be sure how much money was wasted -- or that it won't happen again -- without a more comprehensive investigation.

Cliff DeLouis, Dunna's predecessor as technology chief, pushed last year for MARTA to conduct a five-year audit of all technology spending. He did so, he said, after finding several instances of missing equipment, unnecessary maintenance payments and runaway costs for outside services.

"I could not account for a lot of equipment. . . . I discovered equipment sitting around that I could not tie to specific projects," DeLouis said in an interview. "All these things raise your eyebrows."

In an August memo pushing for the audit, DeLouis warned that payments for equipment and services "may be costing the authority millions of dollars in useless expenditures." In a follow-up e-mail to MARTA's top managers in October, DeLouis suggested the spending "may even be illegal."

MARTA began the audit late last year with the focus on cellphones and telephones. DeLouis said he agreed to scale back the investigation with the hope he could persuade auditors to go deeper later.

DeLouis never got the chance, because he resigned in January. He said he did so after his bosses told him to resign -- or be fired -- because of the way he handled an unrelated disciplinary matter.

DeLouis believes the real reason was his insistence on the audit. "I think it created sufficient friction," he said.

MARTA officials declined to discuss DeLouis' resignation because of a confidentiality agreement signed with him.

Ford, who became general manager in late 2000, said he had never attempted to keep the auditors from asking potentially embarrassing questions. He said he told them to look at "whatever needed to be looked at . . . and turn over every rock."

MARTA's chief counsel, Michael Sloan, said Friday that auditors may still extend their investigation to cover other transactions. "If there is any indication that the audit needs to be expanded for whatever length of time, MARTA will do so," he said.

Future of MARTA

Ford said he has put several management reforms in place to tighten controls over technology spending:

• Large technology purchases face the same scrutiny as big-ticket bus and rail purchases. That means a more stringent review by a committee of managers.

• An internal auditor with an information technology background is assigned to each IT audit.

• The technology chief reports directly to Ford rather than a deputy.

Ford also said he would not sign off on IT spending that was not part of a long-range plan. "Because with IT, what I've seen historically is, the sky is the limit with those folks," he said. "They'll have you on the cutting edge, and MARTA doesn't necessarily need to be on the cutting edge."

In the past, Ford said, MARTA executives often did not even understand IT spending. When IT asked permission to buy more equipment, he said, supervisors would say, " 'OK, here. Here you go. You need it. It's technical stuff. Nobody understands it any damned way when you guys are talking.' So it gets purchase-ordered and off people go and they buy it."

Ford said he was considering whether it would be cheaper and more effective to contract with an outside tech firm.

Gregor, the board chairman, thinks Ford is moving in the right direction.

"He inherited a number of problems with that [IT] unit," Gregor said. "He talked to me about getting better controls."

Ford said, one way or another, he will cage the technology tiger.

"I feel confident that as we go forward," Ford said, "some of the mistakes that were made in the past, we'll learn from them."

Atlanta Journal

-- Anonymous, June 23, 2003