Recently there’s been a lot of writing about “Big Data”. Many organizations and consultancies have begun to speak, write, and offer view points on big data. To me, big data is nothing more than the marketing of analytics that has been going in many organization for severals years/decades. For example, banking, health care, pharma, insurance, and retail have all leveraged some form of big data analytics to support their business decision making for several decades. Most of us were pretty unaware of this, but have increasingly become aware as personal information data breaches have revealed how much information these organizations actually collect.
So why the focus now?
Hard to tell really, but I suspect that there are a few factors at play:
- Open Data: One of the components of big data is the ability to assemble data from different (disparate) sources together and create new levels of insight. This would not be possible without open data (i.e. “free as in beer” data). This often comes from public agencies in the Western hemisphere, but increasingly we’ve seen some companies make some of their data available openly.
- Computational power and storage costs: Moore’s law being in effect, we are at point with the rate of improvement on computational power, mixed with the rate of cost reduction means that we now have more cheap computational power than ever before, and thus more opportunities to experiment with this power, or run analyses that were impossible before. At the very least, you can see examples such as what Microsoft has done with Excel and Access as representative of this trend (without commenting on cause & effect).
- Broader focus on analytics: In the last 10 years, we’ve seen a greater focus on analytics across the board (and this may be linked to the above too).
So what’s interesting about Big Data
To me, the buzzwords are not interesting. What I find to the be the most interesting piece with big data is the open culture that is created and the ability to “mash” otherwise disparate data. I believe this is creating a lot of junk analyses out there (i.e. falsely implying causality because of coincidence, as opposed to truly linked causality). However I do believe that these junk analyses will help augment the dialog about the use of data and analytics, but also help those of us who can demonstrate deeper and thorough appreciation of analytics and data, contextualized in practicality and pragmatism of meaning to provide distinctive answers and help drive new idea generation.