More thoughts about CIOs and agile software development

In putting together a recent print story about CIOs co-opting agile software development techniques as a way to accelerate their ability in the executive suite to evaluate, acquire and deploy new technologies, I came across more information and good points than I could fit in the original print story.

Two of the more valuable points I had to leave out were made by Richard Cheng, a managing consultant who leads the agile practice at Excella Consulting. They both relate to how the agile mindset differs from traditional decision and project management thinking. Those differences lie at the heart of what makes agile such a powerful concept and so tricky to embrace in government.

The first point underscores how a fundamental part of the agile approach is ordering activities by value. Instead of treating all features of a new system or all facets of an IT decision as equals, agile prioritizes tackling the highest value tasks first. This approach has a very convenient benefit in the government space.

“By focusing on the most important, if the funding gets cut or if you have to deploy early, you have the highest priority items already done,” Cheng said. “In the old system, if you cut a two-year project two-thirds of the way through, you may have a big stack of documents and very little else.”

The other point relates to failure, and the toxic nature of that term in government. With agile, “it is important to identity critical failure versus exploratory failure,” Cheng said. “Agile pushes for exploratory failure. If you have to fail, fail early. You also need kill points in the process. One of the problems with the way government works today is that killing something becomes synonymous with failure. People become so invested in certain paths that sometimes things that should be killed are continued.”

Cheng elaborates on these issues in an excellent blog he writes called One More Agile Blog. In a recent post he talked about some of the obstacles government executives face in trying to adopt agile practices. The fundamental challenge is that agile is built around delivering results while the government focuses on managing risks. The disparity in approach is also illustrated when looking at the agile manifesto, a simple four sentences penned by some agile software developers to summarize their principles. They are:

• Individuals and interactions over processes and tools.
• Working software over comprehensive documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan.

As Cheng points out, “The agile world focuses on the left over the right. However, in today’s world the federal government largely operates on the right side of these statements.”

For a more in depth take on how agile principles can be applied to acquisition processes, it’s worth reading chapter 6 of the Defense Science Board’s March 2009 report on Policies and Procedures for the Acquisition of Information Technology. That report’s findings became a key part of the fiscal 2010 National Defense Authorization, which gives Defense officials a July deadline to come up with new acquisition processes that can deliver IT systems in no more than 18 months by incorporating these agile principles.

Also, the report’s authors outline in the executive summary why using techniques like agile to speed up acquisition is critical.

“The deliberate process through which weapon systems and information technology are acquired by DOD cannot keep pace with the speed at which new capabilities are being introduced in today's information age — and the speed with which potential adversaries can procure, adapt and employ those same capabilities against the United States."

DOD officials do have a bit of a head start on agile than their civilian colleagues. They have talked for at least a decade about using a cousin to agile called spiral development for selected software projects, though the intentions have translated into only a handful of actual successful efforts to date.

But there are some examples of agile in the civilian side. Sanjeev “Sonny” Bhagowalia, chief information officer at the Interior Department, answered federal Chief Information Officer Vivek Kundra’s call to the CIO Council to build Data.gov as a public warehouse of raw government data — and do it as quickly as possible. How to do it? Agile, of course.

FCW senior editor Matthew Weigelt interviewed the CIO when he won a 2010 FCW Fed 100 award for his work. Bhagowalia explained the process they used. Notice how agile works in action:

“We brought in the General Services Administration, the Interior Department, the Environmental Protection Agency and the Office of Management and Budget.

"We gave people a task and made them focus on key deliverables and then kept Vivek in the loop. We had daily 30-minute calls to see how things were going. The communication, focus, commitment and passion that people brought to Data.gov are the reasons for its success.

"The idea was to focus exclusively on a small set of requirements and leverage existing acquisitions, such as GSA contracts. In the government, you’d typically wait a year or two to come up with the perfect set of requirements and then do an acquisition. We decided to think in terms of days and weeks, not months and years.

"We rolled out our first site on May 21, 2009. That was almost within one and a half, two months, which is unheard of in the government.”