Ice cream sales and drowning deaths climb together every summer. Nobody sensible thinks the second scoop drags swimmers under. Yet swap those two columns for two from your own warehouse, revenue and email opens, or churn and support tickets, and the same mistake gets made in meetings every week. Two lines move together on a chart, and someone at the table announces that one is driving the other. Sometimes they are right. Often they are not, and the cost of being wrong is a budget spent on the wrong lever.

This part is about the gap between two things moving together and one thing causing the other. It is the single most useful piece of judgement an analyst carries, and it is the one that separates a report people can act on from a report that quietly points them the wrong way. Everything here builds on the statistics from Part 11, where you met samples and uncertainty. You do not need a single new formula, just a slower reflex before you say the word caused.

What it means for two columns to correlate

Correlation is a plain idea dressed in a technical word. Two columns are correlated when they tend to move together in a predictable way. When one goes up and the other tends to go up too, that is positive correlation; taller people tend to weigh more, and spend tends to rise with income. When one goes up and the other tends to go down, that is negative correlation; the colder it gets, the lower ice cream sales fall. When there is no reliable pattern at all, the two are uncorrelated, and knowing one tells you nothing useful about the other.

The picture that makes this concrete is the scatter plot, one dot per row, one column on each axis. A strong correlation shows up as dots that hug an imaginary line; a weak one shows a loose drift; no correlation shows a shapeless cloud. Before you compute a single number, plot the two columns and look, because the shape of that cloud tells you more than any summary. The three panels below are the shapes you are training your eye to recognise.

Reading the correlation coefficient

To put a number on that shape, analysts use the correlation coefficient, written as r, which for the common case measures the strength of a straight-line relationship. It runs from -1 to +1. A value of +1 is a perfect upward line, -1 is a perfect downward line, and 0 means no linear relationship at all. The sign tells you the direction, and the distance from zero tells you the strength: the closer r sits to either end, the tighter the dots cling to a line. The table below is the rough reading I use when someone quotes an r at me.

A rough field guide to r. Treat the bands as guidance, not law; what counts as strong depends on your field.

Two warnings come attached to that single number. The common coefficient only sees straight lines, so a strong curved relationship, spend that rises then falls with age, can post an r near zero while the scatter plot screams that something is going on. That is why you plot first and compute second. The coefficient is also easily yanked by a handful of outliers, so one freak customer can invent a correlation that is not really there, or hide one that is. The number is a summary, and like every summary from Part 11, it can mislead when you stop looking at the data behind it.

Why a correlation is not a cause

Causation is a stronger and more specific claim than correlation. To say that A causes B is to say that if you reached in and changed A, and changed nothing else, B would move as a result. That word intervene is the heart of it. Correlation is something you observe by watching; causation is something you can only confirm by acting, or by an argument careful enough to stand in for acting. A correlation is compatible with a causal story, but it is compatible with several other stories too, and until you rule those out you have a lead, not a conclusion.

When two columns A and B correlate, there are really only a few explanations, and it pays to have them memorised so you can run through them like a checklist. Either A genuinely causes B, or B causes A and you have the arrow backwards, or some third thing quietly drives both, or the whole thing is coincidence. The flow below is the exact set of questions I walk through before I let a correlation turn into a recommendation.

flowchart TD
  A[A and B move together] --> B{Could a third thing drive both?}
  B -->|Yes| C[Confounder, the link may be spurious]
  B -->|No| D{Which way could the arrow point?}
  D -->|B might cause A| E[Reverse causation, test the other direction]
  D -->|A might cause B| F{Ruled out chance and confounders?}
  F -->|Not yet| G[Treat as a hypothesis, design a test]
  F -->|Yes, by experiment| H[Now you may say cause]

Four reasons two columns move together

Give each of those explanations a name, because naming them is how you catch them. The first is the one you hope for: A really does cause B, and an intervention would confirm it. The second is reverse causation, where B causes A and the arrow runs the opposite way to your instinct. Analysts see this often with support tickets and churn; it is tempting to say tickets cause churn, but unhappy customers who are already leaving raise more tickets on the way out, so churn is partly causing the tickets.

The third, and the most common in real data, is a confounder, a hidden third variable that drives both columns and manufactures a correlation between them even though neither touches the other. This is the ice cream and drowning case, where hot weather lifts both. The fourth is plain coincidence, which sounds unlikely until you remember that a dashboard with fifty metrics has more than a thousand possible pairs, and by chance alone a few will correlate strongly while meaning nothing. Test enough unrelated columns and you will always find a spurious match; that is arithmetic, not insight.

The confounder, the trap that catches everyone

A confounder is worth a section of its own because it is behind most of the false causal claims you will meet, and because it hides so well. Formally it is a variable that influences both the supposed cause and the supposed effect, so it props up a correlation that has no direct link inside it. Once you know to look for it, you start seeing it everywhere: cities with more police have more crime, because population drives both; people who take a certain supplement live longer, because the kind of person who takes supplements also exercises and sees a doctor.

The defence against a confounder is to hold it still. If you suspect that engagement is confounding your feature and churn link, compare feature users and non-users at the same level of engagement, casual with casual, power with power. When you slice the data by the confounder and the relationship shrinks or vanishes, you have found your culprit. This is also where a genuinely unsettling thing can happen: slicing by the right variable does not just weaken a relationship, it can flip it completely. That reversal has a name.

Simpson’s paradox, when the trend flips

Simpson’s paradox is a confounder at its most dramatic: a trend that holds inside every subgroup reverses when you pool the groups together. It is not a trick of bad data; the numbers are all correct, and that is what makes it dangerous. Suppose you test two onboarding flows and measure how many new users activate. Split the users into casual and power, and Flow A wins in both groups. Combine everyone, and Flow B wins overall. Both statements are true at once, and which one you report can send the product in opposite directions.

Flow A wins with casual users and with power users, yet Flow B wins overall. The reversal is real, not a rounding error.

The reason is that the groups were not the same size across flows. Flow A was mostly shown to power users, who are harder to activate, while Flow B mostly landed in front of casual users, who activate easily. The user mix is the confounder, and pooling lets an easy audience carry Flow B past a flow that is actually better for everyone. The picture below shows the shape of the reversal: each group trends one way, the combined line trends the other.

How to earn the word cause

If correlation cannot prove cause, what can? The cleanest answer is the controlled experiment. Split similar units at random into a group that gets the change and a group that does not, then compare. Randomising is the magic step, because it scatters every confounder, the ones you thought of and the ones you did not, evenly across both groups. Whatever difference is left is caused by the change, because the change is the only thing that was not shared. This is the logic behind the A/B test, which the series covers in full later on when we design experiments.

Often you cannot experiment; you cannot randomly assign people to smoke, or randomly hand some customers a recession. Then you build the case the way careful researchers do, by stacking evidence rather than finding one proof. Does the supposed cause come before the effect in time? Does more of the cause bring more of the effect, a dose-response pattern? Does the link survive when you control for the obvious confounders by slicing the data? Is there a believable mechanism for how one would produce the other? No single answer settles it, but when several line up and the alternatives keep failing, you can speak about cause with earned, stated confidence rather than a leap.

Ask what else changed before you say caused

Here is the habit to carry out of this part. When two things move together, treat it as the start of an investigation, not the end of one. Run the checklist: could the arrow point the other way, could a third thing be driving both, could this be one of the coincidences that a wide dataset throws off for free. Only when you have knocked those down, ideally with a randomised test and otherwise with stacked evidence and controlled comparisons, do you let yourself write the word caused. Everything before that is a lead, and calling a lead a conclusion is the fastest way for an analyst to lose the room’s trust.

You now have the reflex that keeps correlations honest: read the coefficient but plot the data, name the four explanations, hunt the confounder, and watch for the reversal that Simpson’s paradox can hide in a pooled number. Part 13 turns to showing your findings, choosing the right chart and steering clear of the ones that mislead, which is the visual cousin of everything you just learned about not fooling yourself. Keep the Data Analyst guide open as your map, and revisit Part 11 whenever a number needs its uncertainty stated.

This week, take one claim from a recent report where two things moved together and someone said one caused the other. Write down the confounder that could explain it and the reversed arrow that might be true instead. Then decide what test would settle it. That small exercise, done a few times, rewires how you read every chart you meet.

References

• Interpreting Correlation Coefficients, Statistics By Jim
• Simpson’s paradox, Wikipedia
• Correlation, Causation, and Confounding, Statology

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts