There is data and there is data

All of the data simply shows statistical trends and not predictions...unlike scientific data when applied to the Scientific Method. As the article states when is comes to the comparison with Einstein: "In fact, there is no room for imagination, for serendipitous connections to be made, for learning new things that go beyond the data."
 

"What would big data think of Einstein?"

by

Sydney Finkelstein

June 13th, 2013

BBC NEWS

A friend of mine recently remarked on the uncanny ability of Netflix to recommend movies that he almost always finds interesting. Amazon, too, barrages email inboxes with book recommendations, among other things. Indeed, the entire advertising industry has been transformed by its ability to use data to target individual consumers in ways unimaginable in the Mad Men era.

The power of big data goes far beyond figuring out what we might want to know. Big data helps pharmaceutical companies identify the attributes of their best sales people, so they can hire, and train, more effectively. Big data can help predict what songs are likely to be hits, which wine vintages will taste better and whether chubby baseball pitchers have the right stuff.

But big data should not be confused with big ideas. It is in those ideas — the ones that make us conjure up the image of Albert Einstein — that lead to breakthroughs.

The benefits of big data are so, well, big, that there’s no going back. Yet I don’t need to re-read George Orwell, or scan the latest headlines about the massive snooping of personal communications orchestrated by the National Security Agency in the United States to feel at least some discomfort with big data’s side effects. One that seldom gets notice: in a world where massive datasets can be analysed to identify patterns not easily identified using simpler analogue methods, what happens to genius of the Einstein variety?

Genius is about big ideas, not big data. Analysing the attributes and characteristics of anything is guaranteed to find some patterns. It is inherently a theoretical exercise, one that requires minimal thought once you’ve figured out what you want to measure. If you’re not sure, just measure everything you can get your hands on. Since the number of observations — the size of the sample — is by definition huge, the laws of statistics kick in quickly to ensure that significant relationships will be identified. And who could argue with the data?

Unfortunately, analysing data to identify patterns requires you to have the data. That means that big data is, by necessity, backward-looking; you can only analyze what has happened in the past, not what you can imagine happening in the future. In fact, there is no room for imagination, for serendipitous connections to be made, for learning new things that go beyond the data. Big data gives you the answer to whatever problem you might have (as long as you can collect enough relevant information to plug into your handy supercomputer). In that world, there is nothing to learn; the right answer is given.

I like right answers as much as the next guy, but in my experience those answers are just not enough to motivate people to action. For instance, knowing that email follow-ups to sales calls are the most time efficient is nice, but that fact is unlikely to convince a salesperson who has always picked up the phone to change his approach, especially if his approach has always worked for him.

People don’t think in the same way that data behaves. They need to be convinced, they want to be part of the creation of the solution. They don’t like the solution to be imposed on them. You can have all the “optimal” solutions you like, but in the real world managers need to convince other people to execute on those solutions. And people have a habit of wanting to contribute to the development of the solutions.

In business, big data doesn’t necessarily drive out creativity; it’s just that its scientific imprimatur makes it very hard to argue the opposite way. Yes, it is possible for creative people to start further down the field when they have a deeper understanding of the underlying relationships that govern their discipline. Advertisers can design better campaigns if they truly understand what consumers are buying and why. But sometimes you need to break the rules to create anything new. Apple’s original iPod was such a hit precisely because it emphasised simple and elegant design features rather than what everyone else was competing on — MP3 sound quality.

Just as companies that build their business on “best practice,” ensure that they will never do more than anyone else, companies that let big data dominate their thinking and management style will not be the ones who change the rules of the game in their industry. Even in the leading repository of big data thinking — Silicon Valley — how many start-ups have taken form specifically to capitalise on big data insights? Not Facebook, not Google, and definitely not Apple. These companies actively leverage big data to grow their businesses, but the spark that led to their creation was personal, entrepreneurial and even idiosyncratic.

The inability to understand or capture the human element — that personal, even idiosyncratic, thinking that drives genius — in business is the biggest danger that comes from big data. Has there ever been a major breakthrough whose origin doesn’t reside in the brain of a man or a woman? Imagine in the not-too- distant future a brilliant person, a genius, proclaiming a new way of thinking that is contrary to big data. What would happen to her ideas if she bucked the orthodoxy of big data to suggest a different view of the world not consistent with the dominant digitally derived solution? We might lock up her ideas. If anyone paid attention to what she said, she would be denounced as uninformed.

Companies, like civilisations, advance by leaps and bounds when genius is let loose, not when genius is locked away and deemed too out of the mainstream of data-driven knowledge.

What if Albert Einstein lived today and not 100 years ago? What would big data say about the general theory of relativity, about quantum theory? There was no empirical support for his ideas at the time — that’s why we call them breakthroughs.

Today, Einstein might be looked at as a curiosity, an “interesting” man whose ideas were so out of the mainstream that a blogger would barely pay attention. Come back when you've got some data to support your point.