Case Study: Best Practices for Backup and Recovery from Shaw Air Force Base

SPECIAL REPORT: Storage Management

By Barbara DePompa, 1105 Government Information Group Custom Media

Best Practices for Backup and Recovery

DriveSavers officials recommend government organizations take necessary precautions to maintain important data, and can recover that data in the event of a disaster. Basic backup tips include:

*Invest in redundant backup systems.

*Establish a structured backup procedure, using software compatible with the operating system and applications, to make copies of all critical data files.

*Periodically test backups to verify that data – especially databases and other critical files – are being backed up properly.

* Keep at least one verified copy of critical data off site.

When disaster strikes, DriveSavers officials suggest the following:

*If possible, backup the data immediately.

*If the drive makes scraping, tapping, clicking or humming sounds do not use utility software.

*Do not power up a device that has obvious physical damage or is making unusual sounds.

*Shut down the computer to avoid further damage to the drive and its data.

*Do not attempt recovery on severely traumatized drives, such as turning the computer off and on, using over-the-counter diagnostic tools. This may cause further damage or permanent data loss.

*Configure another computer/server to temporarily replace the problem unit, restore available backups onto the new unit and reconfigure it as necessary to begin productive work.

*Contact a service provider, such as DriveSavers, for recovery advice. Because of the broad range of complex operating systems, turning to utility software alone can potentially cause data loss.

When Shaw Air Force Base in South Carolina experienced an unpredictable backup failure, compromising access to every file on the squadron's 12-drive RAID server, the systems administrator turned to a backup and recovery services provider to rescue the base's valuable data.

At risk was crucial data backed up from 350 computers.  Archives for the base housing office were at risk of permanent loss. This is information that Congress mandates to be kept in perpetuity. Vital training data, including fire safety as well as flight readiness records were also no longer accessible. Without these vital records, Air Force students would be required to repeat training, which would have caused delays and increased training costs as well.

“The crash happened in stages, and I thought that I could initially solve the problem myself,” said Dean Johnson, systems administrator for the Civil Engineering Squadron at Shaw AFB. “But, when the Adaptec system controller failed, I stopped my recovery efforts immediately. I knew that if I fired up a new controller and it started a new array, I could kiss my data goodbye.” 

After a recommendation from Adaptec system officials, Shaw AFB turned to DriveSavers for help. The drives were sent to DriveSavers and analyzed in its certified ISO 5 clean room. While the storage subsystems was initially thought to be a RAID 5 array, which would have allowed rebuilding of the system using data stored on a spare drive, it was discovered that that the drive subsystem was actually configured as RAID 0 with data striped across all 12 drives.  DriveSavers' engineers were able to unravel the challenge of the unknown order of the drives, and recover a full 45 GB of the most-wanted data. “The value of the recovered data far exceeded the cost of recovery,” said Johnson.  “If the data wasn't recovered, we would have been illegal in the eyes of our major command, and eventually it could have gone up to Washington, D.C.”

DriveSavers has successfully recovered critical data for government agencies including Lawrence Livermore National Laboratory, NASA Goddard Space Center and the Smithsonian Institute.