Usually, I am the first person to reject a new fad and it is quite possible that I am misreading what “Big Data” is all about.  However, the more I understand – once the hype is taken away – the more I come to realize that Big Data is here.  This post is to provide a brief overview of what I have found and believe.

Between the beginning of mankind and 2003 the world has accumulated somewhere between 3 to 12 exabytes (exabyte = 1 million terabytes) of data according to David Houle, author of Entering the Shift Age.  By 2010, we create in aggregate 3 exabytes every four days.  Information overload, anyone?

All of this data is not useful, but it is big.  The key will be to collect only those data that are useful.  I believe that this will bring in a new era of new knowledge.  Yes, a lot of it will be waste at the beginning.  However, as I look to the future and progress on collecting data we would previously have categorized as “unknown and unknowable.”

If you remember, it was W. Edwards Deming that spoke of data that are “unknown and unknowable” – meaning we really don’t have the data we need in organizations to make good decisions.  Things like what is the price of losing a customer and how would you measure it?  What is the cost of a disgruntled worker?  These would be categorized as “unknown and unknowable.”

I am not saying that we will have the best data to make 100% decisions, but as more data are collected in the Shift Age there will be additional shadows cast on data we did not have before.  Houle sees the rise of new jobs that are data purely data focused.  This is probably true, but how we go about finding and collecting the right data still seems the most worthwhile path.

What are your thoughts on Big Data?

Tripp Babbitt is a service design architect and organization futurist.  His organization helps service organizations understand future trends, culture and customer and design organizations based on new knowledge.  Read his column at Quality Digest and his articles for CustomermanagementIQ.com. Reach him on Twitter at www.twitter.com/TriBabbitt or LinkedIn at www.linkedin.com/in/trippbabbitt.