Big Data 2.0
- More of Big Data 1.0: increase in efficiency of processes, etc.
- Big Data from sensors, Internet Of Things.
- Connecting data from different sources (weather, traffic) and finding correlations.
- Data turns into knowledge: no more data, rather knowledge in context. Imagine: Newton 's law of motion: F = ma and table representing values of force and respective change in velocity. F = ma is knowledge, the table is data.
- Biggest change: Big Data 1.0 has been about finding correlations: you feed data and the computer discovers patterns that are nothing more than mathematical / logical relations between variables; no "theories" - that explain why they hold true. Big Data 2.0 would be more like science. Big Data 2.0 will focus on constructing theories of why the relations hold true. (Consider, A/B testing: you try to figure out which of the two your customers like more without any understanding of why they do so. Big Data 2.0 is going to change that.)
- Once you have theories, you can build on those theories, just like scientific theories are build upon one another.
- Consider what happens when you connect theories built with data from different sources. You have theory / model that relates "traffic" with "weather conditions, day of the week etc". You have another theory that makes weather forecasts based on models. If you connect the two theories / models, you can predict traffic. Up until now it has been like: weather => traffic, weather history => weather forecasts. In the age of Big Data 2.0, it's going to be like: weather history => weather forecasts => traffic. The world becomes more predictable.
- When you have general theories of complex systems, you feed data and wait for the computer to tell you interesting things from data.
- Finding patterns in a uniform data source is not going to help you build theories. You have got to connect different data sources.
- Big Data 2.0 will be more like: finding correlations and then explaining the correlations logically (this is why it holds true) and then experimenting more to find out whether the hypothesis was correct. And in this way coming up with theories. Theories which would later be used by others to build upon.
- Big Data 2.0 would form the foundations for "new kinds of sciences": Complex Systems, Biosciences, molecular and nano sciences, Social and Political Sciences, business and management, Culture, Education, Brain Sciences, Atmospheric and Earth Sciences.
- But never underestimate the role that Big Data 1.0 has played. Previously, all our data were siloed in data storage devices with no way to query or make sense of all the data. Big Data 1.0 and tools like Hadoop have provided us with ways to make sense of all the data.
- But up until now it has been just correlations inside a data source and no relation outside a particular data source and no relation among data sources. It's all going to change with the advent of Big Data 2.0.
No comments:
Post a Comment