For the past five decades...

...we’ve witnessed the geometric growth of computing power, and the associated geometric reduction in cost. Not only has processing power doubled at this relentless rate, so have storage and our application of computers in our everyday lives. Five decades ago computers were massive machines that did nothing but compute mathematical equations. Today computers that fit in the palm of your hand can help us find the closest taco stand.

With the increases in computational power, storage, and the ability to automate data collection, the world constantly collects more data. IDC (International Data Corporation) released a study in May 2010 that projected a 45-fold annual data growth by the year 2020. IDC continues to study the digital universe. In the updated 2011 study, IDC projected that the amount of information created and replicated would surpass 1.80 zettabytes (1.8 trillion gigabytes) by 2015. Data is growing faster than ever before and by the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet. With the advent of cloud computing, digital delivery of videos, music, and business data, that projection may turn out to be a bit conservative.

As more people use smartphones, we create even more data. In 2015, over 1.4 billion smart phones shipped - all packed with sensors capable of collecting all kinds of data, not to mention the data the users create themselves. All those smart phones took 1 trillion photos with billions of them shared online using Facebook. Some predict that nearly 80% of photos this year will be taken on smart phones. By 202 there will be over 50 billion smart connected devices in the world, all developed to collect, analyze and share data.

Think on this, less than 0.5% of all data is ever analyzed and used.

The collection, transmission, and storage of data continues to explode. Is this what everybody means by big data? Perhaps. Clearly, in our rapidly digitized world, we’ve become more dependent upon technology and the data that drives it. But that still doesn't answer the question, "What is big data?"

Big data got much hype between 2012 and 2015. In that time the term showed up on wish lists of many business and IT leaders' capital and project budgets. The term appeared in just about every forward looking article about the future. Futurist writers and speakers put it on their Top Ten lists of disrupters.

What happened? Well, the shiny newness of big data passed.

Perhaps big data slid past the peak of Inflated Expectations and is somewhere on the Slope of Enlightenment. The Gartner chart above predicted in 2012 that big data would reach the Plateau of Productivity in 2 – 5 years. We see evidence of big data systems making waves.

In the articles here, we explore what big data is, and how the convergence of the Internet of Things, increased automation, listening devices and video capture help create the information that becomes big data.