Big Data, Small Data, Everything Data.

It was a year ago ‘big data’ was starting to emerge as one of the Advertising & Marketing industry’s biggest buzzwords. In my post ‘From Big Ideas to Big Data’, I briefly touched on the possibility of a talent shortage as a big challenge faced by companies moving  to realise the value of Big Data. But more importantly, I discussed the fundamental need to balance between creativity, gut and reliability of data. After all, lots of data do not necessarily translate to actionable insights.

So fast-forward a year later, today, the common theme that runs through every industry conference is around Big Data. Most keynote speakers find every opportunity to talk about big data, small data, everything data. Machines and the data that comes with it is monopolising our world, if it hasn’t done so already.

Being human in a data-driven world.

For the first time in history, we are living to keep up with both time and technology – human constructs which are not on our side. Through out human evolution, it has taken many generations for our genetic code to adapt to changing environments and circumstances and as a result, our well-adapted genome has lasted for hundreds of thousands of years.

Interactive timeline of human evolution

However, in the span of the last couple of decades, we have seen unprecedented change in our world and lives – and these changes are only accelerating. We are now attempting to thrive in a digital world that we weren’t exactly programmed for.

Big data disrupts the fundamental human makeup. 

It is our brain’s inability to reason and act intelligently in the face of analytical data.

We have the most remarkable organ in the body – the human brain. With a relatively small mass of 1.4 kilograms, it produces our every thought, action, memory and feeling we experience in the world. Our brains form a million new connections for every second of our lives and its pattern and strength is constantly changing with no two brains alike. It is with these changing connections that memories are stored, habits learned, and personalities shaped.

But what we know now is: there are two different ‘systems’ underpinning our brain activity and the system that works fastest (System 1) happens to be the most unfavourable for interpreting data. In the book Thinking Fast and Slow by Prof. Daniel Kahneman he describes System 1 as automatic, continuous and cannot be turned off – generating impressions, intuitions, intentions and feelings, while System 2 is a slower, more reason-based approach that is conscious, considers evidence and questions assumptions.

Although we fancy ourselves to be primarily System 2 creatures, many of our mental operations are System 1 in nature. We are driven by System 1 because it is instinctual and we are more likely or more intuitive to make quick decisions with little mental energy.

Our mind is strongly biased and we easily jump to conclusions. 

People are prone to apply causal thinking inappropriately, especially in situations that require statistical reasoning. The confidence that individuals have in their beliefs depends mostly on the quality of the ‘story’ that they can tell about what they see, even if they see little. Once an association has been created (causal link) our brain latches for possible causes. This is because System 1 works on associative memory and any weak stimulus is sufficient to bring it over threshold. Any ambiguity is suppressed as one interpretation is applied. Unfortunately System 1 does not have the capability of reasoning. Prof. Daniel Kahneman calls this phenomenon “the illusion of validity.”

An example of associative machinery and how System 1 works:

A B C or 12, 13, 14?  You may notice that the B and 13 are identical.

Everything can be made coherent. In the context of letters, the ambiguous stimulus is going to be read as a letter, and in the context of numbers, it is going to be read as a number. The brain will generate associatively coherent representation or reactions to situations.

We search for evidence that upholds our beliefs than seek to disprove them; even if the beliefs are illogical.

We often ignore statistical information in favour of our own predictions based on familiar traits. Everything we see, experience, think and feel can be adjusted to fit with our beliefs. The dangerous part here is our version of reality is in fact a creation of our beliefs and any obsolete belief system can still influence how we evaluate everything in our lives. And once set, our minds will construct any argument to support it. We know this as “confirmation bias“.

Though I have only provided with two examples, we need to be aware there are hundreds more biases we need to be aware of.  It is indeed a fascinating subject. I found this interesting article that lists down the 12 cognitive biases that prevents us from being rational which I believe are highly applicable when it comes to handling data in the business world.

Dangers Ahead.

Technology has transformed businesses to the point that hard numbers are now the driving force in business decisions.  It is the rational and analytical that are driving decision making – however, data lacks human relevance. Human involvement is only integral to the first and final steps – selecting the metric and then interpreting the statistical correlations. Yet in these steps, we have seen from above the natural biases that occur when we interpret data, reducing any possibility of reaching a good outcome. We should not be too reliant on Big Data as the solution for everything that will pave the way the business operates. If anything, more automation might come out of Big Data, not insights.

Going in for the long  haul. 

Irving Fisher’s biggest mistake was that he believed that the world is moved by numbers, rather than emotions – Tim Halford

What I find most interesting for human involvement is the messy middle step of asking what the right questions are, but in theory, this step has been eliminated by software that analyses all the possible correlations and scores and presents them accordingly. What we need to do is learn to control our System 1 and let our System 2 kick in more often. In the enormous pool of data, we need to take the time to understand the underlying logic.

Let’s remind ourselves that we are not passive consumers of data and technology. Data and data sets are not objective; they are creations of human design. We shape the role it plays in our lives and the way we make meaning from it, but to do that, we have to pay as much attention to how we think as how we code. We have to ask hard questions to move past counting things to understanding them.

Data causes challenges of interpretation. Data alone is information – who, what, when. There is no inherent value in any piece of data because it could mean anything. Therefore, having too much data can be meaningless. The only way we make meaning is the manifestation of our memory, our history.  It’s the ‘Why’ that gives emotional context and the willingness to entertain the possibility of being wrong. It is the same process as sensors delivering data to the brain, and it is the brain that evolves and assembles that data into information that is meaningful to a human being. So any form of data, whatever we would like to call it, if left unused or misused becomes an end in its own right.

So as the Advertising & Marketing industry focuses on how large datasets can give insight on previously intractable challenges, I will continue to ask the question: When can we move from the focus on merely Big Data towards something more three-dimensional: data with depth?

At Google, one of the largest data powerhouses on Earth, has developed a workshop Unconscious Bias @ Work, in which more than 26,000 Googlers have taken part. The purpose of this initiative is to recognise unconscious biases that exist in our day to day jobs and workshops are conducted to highlight four bias busting techniques which may help to mitigate the potentially negative influence on unconscious bias.

The four techniques presented in these workshops are:

1. Gathering of facts
2. Reliance on consistent structure and criteria when making decisions
3. Waching for subtle cues
4. Fostering awareness and accountability