Something to celebrate

When Doug Babb took on the information technology systems at Hill Air Force Base, Utah, the base was running a dizzying array of operating systems, servers and applications.It had Hewlett-Packard OpenVMS, Sun Solaris, Microsoft Windows and varying flavors of Unix and Oracle. Frequent downtime plagued the operation. Each hour of downtime cost about $1 million, and the base had suffered through eight episodes in three months."It was like opening a closet where everything falls out," said Babb, a contractor with Systems Implementers Inc. of Tucson, Ariz. "I kept wondering, do I put everything back in the closet or do I throw it all out and start over?"Forty-two servers in six buildings ran 80 applications. "Everybody had their own administrators, agendas and cultures," Babb said.The confusion caused by the clash of systems even affected morale. Babb described the atmosphere at the base as having "little joy."Downtime wasn't an option for Hill's Ogden Air Logistics Center, which provides maintenance, engineering and logistics management for some of the service's most important aircraft and weapons, including the F-16 Fighting Falcon, the A-10/OA-10 Thunderbolt II, the C-130 Hercules and the Minuteman III intercontinental ballistic missile.Employees who performed maintenance on the aircraft needed access to a wide array of databases and systems, including those that handled scheduling, parts and replacements."The biggest problem was that we had a lot of different hardware and programs," said Mike Jolley, project manager and chief of the operational policy branch at the base's Information Technology Directorate. "They were all built at different times by different people."For 11 months, Babb worked on-site with Jolley and a team of 27 people, including civilian employees and other contractors, to deploy a new system dubbed Project Bonfire.The base had been using HP Superdome and Sun servers. "Hill had huge metal," Babb said. "You name it, they had it." But the Windows environment was unstable, and Babb feared data attacks and had other security concerns, he said.Choosing a new direction for the base's system proved to be fairly easy. "The only vendor out there that had gone through National Security Agency security [certification] was Red Hat," Babb said. "That narrowed down our choices pretty well."Project Bonfire is based on Red Hat Enterprise Linux 5, and it employs grid computing by linking servers to boost performance. Babb said the back of one of the grid servers looks "like veins on a body. It's very complex and interconnects with everything."Base officials also liked the fact that open-source systems have a lower cost of ownership. By choosing the less expensive x86, 64-bit architecture and the Linux platform as its standards, the Air Force saved more than $5 million a year. The Linux system costs $100,000 a year.The base supports 22,000 on-site users and 3,000 remote users. For some applications, Project Bonfire moved from a client/server architecture to a Web-based interface, which required training for some users. But for the most part, the conversion was transparent and users needed no training."One day, they were on the old system, then one day on the new system," Babb said, adding that the cost of training 25,000 users on a new system "would have been beyond belief."One of the reasons the team chose the name Project Bonfire is that "it was a sort of celebration," Babb said. "We consolidated servers but also storage. We worked on [uninterruptible power supplies] and cooling. It was a crazy 11 months."Since the base deployed the system, the turnaround time to equip an aircraft with new wheels and brakes shrank from 38 days to 16 days. Project Bonfire can run its largest data warehouse load in less than three hours, down from 12 hours with the old system. Response times decreased from 10 seconds to a little more than one second. And instead of being housed in six buildings, the system now resides in two.The Project Bonfire team is deploying the same technology at 59 Army sites and has received inquiries from the Marine Corps and Navy."The bottom line is not to create a really cool IT system," said Rich Wood, vice president of Systems Implementers. "The whole idea is to support the warfighter and get them the tools they need."