Big Data, Small Data, Everything Data.
It was a year ago ‘big data’ was starting to emerge as one of the Advertising & Marketing industry’s biggest buzzwords. In my post ‘From Big Ideas to Big Data’, I briefly touched on the possibility of a talent shortage as a big challenge faced by companies moving to realise the value of Big Data. But more importantly, I discussed the fundamental need to balance between creativity, gut and reliability of data. After all, lots of data do not necessarily translate to actionable insights.
So fast-forward a year later, today, the common theme that runs through every industry conference is around Big Data. Most keynote speakers find every opportunity to talk about big data, small data, everything data. Machines and the data that comes with it is monopolising our world, if it hasn’t done so already.
Being human in a data-driven world.
For the first time in history, we are living to keep up with both time and technology – human constructs which are not on our side. Through out human evolution, it has taken many generations for our genetic code to adapt to changing environments and circumstances and as a result, our well-adapted genome has lasted for hundreds of thousands of years.
Interactive timeline of human evolution
However, in the span of the last couple of decades, we have seen unprecedented change in our world and lives – and these changes are only accelerating. We are now attempting to thrive in a digital world that we weren’t exactly programmed for.
Big data disrupts the fundamental human makeup.
It is our brain’s inability to reason and act intelligently in the face of analytical data.
We have the most remarkable organ in the body – the human brain. With a relatively small mass of 1.4 kilograms, it produces our every thought, action, memory and feeling we experience in the world. Our brains form a million new connections for every second of our lives and its pattern and strength is constantly changing with no two brains alike. It is with these changing connections that memories are stored, habits learned, and personalities shaped.
But what we know now is: there are two different ‘systems’ underpinning our brain activity and the system that works fastest (System 1) happens to be the most unfavourable for interpreting data. In the book Thinking Fast and Slow by Prof. Daniel Kahneman he describes System 1 as automatic, continuous and cannot be turned off – generating impressions, intuitions, intentions and feelings, while System 2 is a slower, more reason-based approach that is conscious, considers evidence and questions assumptions.
Although we fancy ourselves to be primarily System 2 creatures, many of our mental operations are System 1 in nature. We are driven by System 1 because it is instinctual and we are more likely or more intuitive to make quick decisions with little mental energy.
Our mind is strongly biased and we easily jump to conclusions.
People are prone to apply causal thinking inappropriately, especially in situations that require statistical reasoning. The confidence that individuals have in their beliefs depends mostly on the quality of the ‘story’ that they can tell about what they see, even if they see little. Once an association has been created (causal link) our brain latches for possible causes. This is because System 1 works on associative memory and any weak stimulus is sufficient to bring it over threshold. Any ambiguity is suppressed as one interpretation is applied. Unfortunately System 1 does not have the capability of reasoning. Prof. Daniel Kahneman calls this phenomenon “the illusion of validity.”
An example of associative machinery and how System 1 works:
A B C or 12, 13, 14? You may notice that the B and 13 are identical.
Everything can be made coherent. In the context of letters, the ambiguous stimulus is going to be read as a letter, and in the context of numbers, it is going to be read as a number. The brain will generate associatively coherent representation or reactions to situations.
We search for evidence that upholds our beliefs than seek to disprove them; even if the beliefs are illogical.
We often ignore statistical information in favour of our own predictions based on familiar traits. Everything we see, experience, think and feel can be adjusted to fit with our beliefs. The dangerous part here is our version of reality is in fact a creation of our beliefs and any obsolete belief system can still influence how we evaluate everything in our lives. And once set, our minds will construct any argument to support it. We know this as “confirmation bias“.
Though I have only provided with two examples, we need to be aware there are hundreds more biases we need to be aware of. It is indeed a fascinating subject. I found this interesting article that lists down the 12 cognitive biases that prevents us from being rational which I believe are highly applicable when it comes to handling data in the business world.
Technology has transformed businesses to the point that hard numbers are now the driving force in business decisions. It is the rational and analytical that are driving decision making – however, data lacks human relevance. Human involvement is only integral to the first and final steps – selecting the metric and then interpreting the statistical correlations. Yet in these steps, we have seen from above the natural biases that occur when we interpret data, reducing any possibility of reaching a good outcome. We should not be too reliant on Big Data as the solution for everything that will pave the way the business operates. If anything, more automation might come out of Big Data, not insights.
Going in for the long haul.
Irving Fisher’s biggest mistake was that he believed that the world is moved by numbers, rather than emotions – Tim Halford
What I find most interesting for human involvement is the messy middle step of asking what the right questions are, but in theory, this step has been eliminated by software that analyses all the possible correlations and scores and presents them accordingly. What we need to do is learn to control our System 1 and let our System 2 kick in more often. In the enormous pool of data, we need to take the time to understand the underlying logic.
Let’s remind ourselves that we are not passive consumers of data and technology. Data and data sets are not objective; they are creations of human design. We shape the role it plays in our lives and the way we make meaning from it, but to do that, we have to pay as much attention to how we think as how we code. We have to ask hard questions to move past counting things to understanding them.
Data causes challenges of interpretation. Data alone is information – who, what, when. There is no inherent value in any piece of data because it could mean anything. Therefore, having too much data can be meaningless. The only way we make meaning is the manifestation of our memory, our history. It’s the ‘Why’ that gives emotional context and the willingness to entertain the possibility of being wrong. It is the same process as sensors delivering data to the brain, and it is the brain that evolves and assembles that data into information that is meaningful to a human being. So any form of data, whatever we would like to call it, if left unused or misused becomes an end in its own right.
So as the Advertising & Marketing industry focuses on how large datasets can give insight on previously intractable challenges, I will continue to ask the question: When can we move from the focus on merely Big Data towards something more three-dimensional: data with depth?
At Google, one of the largest data powerhouses on Earth, has developed a workshop Unconscious Bias @ Work, in which more than 26,000 Googlers have taken part. The purpose of this initiative is to recognise unconscious biases that exist in our day to day jobs and workshops are conducted to highlight four bias busting techniques which may help to mitigate the potentially negative influence on unconscious bias.
The four techniques presented in these workshops are:
- Gathering of facts
- Reliance on consistent structure and criteria when making decisions
- Waching for subtle cues
- Fostering awareness and accountability
3 thoughts on “Big data disrupts the fundamental human make-up”
Hi Anne – thanks for sharing such great article!
On Twitter you said you would be happy to hear my thoughts…
Here we go, a little Thinking out loud…
There are a few very crucial points you make in my opinion in your post. I did not ranked them, interchange their importance at your own will if you want:
• “…the fundamental need to balance between creativity, gut and reliability of data. After all, lots of data do not necessarily translate to actionable insights.”. Touché! What’s the value of an overload of data if you’re incapable of determining its reliability or translate them in actionable insight (or even plain old Action Plans…). I guess our System 2 has somewhat evolved over the past decades to deal with an ever growing base of statistical, structured, categorized data. Though, as you state that’s merely “selecting the metric and then interpreting the statistical correlations”. That doesn’t justify if the data is reliable: once we’ve a bias we’ll find a data set that proves our bias is correct. The ‘depth’ you refer to at the end, to me, comes from mixing in unstructured data. Articles, like this one, which can add body for or against a bias, and help to find actionable insights.
• “We have the most remarkable organ in the body – the human brain. … Our brains form a million new connections for every second of our lives… It is with these changing connections that memories are stored, habits learned, and personalities shaped.”. Our brain is not only remarkable, it is a complex system. There’s a famous quote from Russell Ackoff – “The performance of a system is not the sum of its parts. It is the product of its interactions”. Projecting that back to businesses, which are complex systems as well because there are many human brains (call them ‘parts’ for the sake of comparison), it means that creation and nurturing of CONNECTIONS is what will make an organization successful, the focus on the interactions between the brains that drive the results. NOT the optimization of a part of the system to satisfy just a subset of the parts (brains).
• Technology, it is a red wire in our modern life: “…we have seen unprecedented change in our world and lives – and these changes are only accelerating.”. System 1 is fast, System 2 is slower. Then you write “What we need to do is learn to control our System 1 and let our System 2 kick in more often”. And this is for me the great contradiction of our time (and maybe a confirmation of the previous point): we need to use a slow system more often in an environment which speed is ever increasing (driven by technology).
Would there be any change we can slow System 1 down a little, let it absorb (as unbiased as possible) some of the unstructured materials. While in the mean time speed up System 2 a little to get the right metrics and the unbiased, reliable data analyzed. And then, most important, cultivate a conscious interaction between System 1 and System 2?
Thanks Patrick for your feedback.
I find this subject very fascinating so am happy you find it too. It was my intention to write this blog post to shed light that many people IMHO (or at least in my industry) have become too fixated on data for decision-making – as if data speaks for themselves. One of the biggest mistakes people make in data-driven decisions is that there is little understanding of causation, and without any theory, plausible correlation is not required. Don’t get me wrong: big data is not worthless, but it can be useless if we do not have a theory. To make data work for us and for us to make better decisions, we need to be aware of the assumptions we make about big data. We need to understand the importance of theory and its methodology when collecting data for analysis. Often answers are quickly obtained through correlation – sure, it is a good starting point but unfortunately, it also ends there. Instead, causation should be the goal.
Then, we must not forget that there is the human element that is in play in our decision making. It does not matter how big the sample is, if it is bias. We need to build this self-awareness and consciously tell ourselves to ‘Hold on. Let’s think about this.’ What I find most depressing about this ‘modern life’ is that time is a human construct – yet we have allowed it to consume us, to dictate the pace of our lives. We have created this stress for ourselves. In this mad rush, we have desperately turned to data to provide us with quick answers.
So whether it is slowing down System 1 or accelerating System 2, one thing is for sure: We need to think things through. And more often. Sure, machine learning is good for large data sets in relatively stable scenarios, but as in life, we find more volatility than stability. We need to remember to apply critical thinking – a trait of any good forecaster is the element of ‘doubt’ – the willingness to entertain the possibility of being wrong. When we become too reliant on machine learning for decision-making we may fail to see the full picture. And it starts to paint a very scary picture when decisions in the boardroom are made based on two fundamental flaws: cognitive biases and sample biases.
I can only agree with you Anne! The challenge I’ve put myself on my blog is to try to give people something to read based on Complex Adaptive Cycles (Panarchy) as theoretic model for thought about how to “manage” a business and making (better) decisions.
I intend to start doing more with that in the next months…. Stay tuned! 😉