Sandy makes the case for big data

If there are any doubters left about the value of big data and analytics, Hurricane Sandy has surely blown them away.

Take a look at the stats our sister publication FCW quotes in its story on the accuracy of computer modeling and weather forecasts leading up to the storms landfall.

Three-days out, forecasters predicted within 10 miles where landfall would occur. Twenty years ago, they may not even have known that the storm would have taken its unconventional turn left.

In 1992, the result would have been much more devastating because there would have been little to no warning.

For a deeper dive into the technology, check out our other sister publication, GCN, and its story on the computer models.

Sandy is on track to be the second most expensive storm in U.S. history after Katrina, with billions in damages and over 100 lives lost.

But thanks to the rapidly growing speed and power of computers, and the ability to collect and analyze data faster and more accurately than before, an even bigger disaster was averted.

A variety of technologies are converging to make this possible, and its application is far wider than weather forecasting, though that's probably the most immediate and valuable benefit.

Our ability to analyze massive amounts of data is catching up with our ability to collect it.

One of the things I’ve always found fascinating about the analytics trend is that the tools used to analyze and visualize data are getting better and better, so that even average Joes have the ability to query and manipulate data in unprecedented ways.

In my thinking, that’s where a great business opportunity lies. While big heavy science and technical users may be the early adopters, the broader and perhaps more lucrative market doesn’t lie with power users.

Instead, it is the managers and operations folks who are making dozens of decisions each day. The challenge and opportunity is identifying their pain points in their daily operations and helping them discover how to use analytics to address those issues. The obvious ones involve health care, intelligence and finance, but there are multiple use cases that are yet to be discovered.

The big data trend is really just in its infancy, and the exciting thing is that the more analytical power that gets put in to more people’s hands, then the better and more accurate decision-making will become.

It won’t stop the next Sandy; but it might save more lives and property.