When we count, we change what counts
In a nutshell
Bias and over-confidence can cloud our judgement when an issue is complex
We can be biased in favour of something when it is presented as a number
Over-confidence in an outcome can stem from the simplicity with which it is presented
I decided to write this note after reading another about the need to eat 30 fruits, vegetables, nuts, seeds (daily or weekly, the claim changed!) to remain healthy. I will write about that separately.
My point here is simply to draw attention to the notion that once an idea is presented as a number, many people treat the idea as fact without questioning where the number came from and whether it is in any way valid.
I did a little research and came up with a couple things that seem to explain this:
Quantification fixation
The illusion of validity
Quantification fixation
Quantification fixation [1] is the tendency to give more weight to whichever dimension of a decision is expressed as a number, even when the underlying information is unchanged. This bias occurs because numbers feel easier to remember (my opinion) and compare.
The effect is stronger when the numbers are easy to process (e.g., 75% rather than 51/68) and in people who feel confident with numbers (not those objectively better at math).
Stating the title of the paper [1] differently, I’d go as far as to say that:
“…when we count, we change what counts…”
Examples of quantification fixation
Step counts – As far as I can tell, 10,000 steps/day is an arbitrary marketing thing. Slow shuffling around a large office or warehouse and strenuous hiking can yield similar step counts whilst ignoring strength, movement intensity, sleep, and outdoor time.
LDL cholesterol as a proxy for heart health – “Bad cholesterol” levels are easy to remember and reduce with a lifelong statin prescription. No matter that LDL is not the cause of heart disease, neither is cholesterol which is carried around in LDL. It’s much easier to remember our LDL measurement than to think about plaque being caused by things like smoking, seed oils, high carbohydrate diets that can cause repeated arterial wall damage.
The illusion of validity
The illusion of validity [2] is the tendency to feel highly confident in a prediction or judgment because the evidence forms a neat, coherent story - even when that evidence is scant, contradictory, or known to be a weak predictor. Confidence tracks how well the story hangs together, not how well it predicts.
Example of the illusion of validity
Regularly eating 30 plants – The example that prompted this article and which I’ll write more about soon. Our meme-driven world loves tidy stories. Gut microbiota, plant diversity and a round number fits right in. “30 plants a week” feels proven, even though it comes from an observational study that can never show cause and effect.
Summary
Quantification fixation and the illusion of validity work in series. A number feels accurate and objective, which lends the story around it unearned confidence.
In other words, a convenient number becomes a stand-in for understanding, then gets repeated as if the underlying facts were known, understood and beyond further discussion.
Previous articles touching upon human psychology
The Extinction of Experience – we accept the unacceptable because it is normal. We lack the perspective needed to recognize when something has gone wrong.
References
Chang LW, Kirgios EL, Mullainathan S, Milkman KL. Does counting change what counts? Quantification fixation biases decision-making. Proc Natl Acad Sci U S A. 2024 Nov 12;121(46):e2400215121. doi: 10.1073/pnas.2400215121. Epub 2024 Oct 28. PMID: 39467152; PMCID: PMC11572960.
Kahneman, D. (2011). Thinking, Fast and Slow. Chapter 20, "The Illusion of Validity,"