0 saved
46.6K views
One of Peter Drucker’s most known quotes is “If you can’t measure it, you can’t improve it.” It’s hard to deny that measurement and gathering data are critical factors to success, which is why Goodhart’s Law can be a harsh reality check.
In the words of anthropologist Marilyn Strathern, Goodhart’s Law tells us that ‘when a measure becomes a target, it ceases to be a good measure.’
GOODHART’S LAW IS EVERYWHERE.
Goodhart’s Law can be seen anywhere where people want to measure progress to reach a goal, for example:
-
A Soviet Union nail factory was famously said to have been given a target related to the number of nails and proceeded to create miniature, useless nails. When the target was shifted to weight, the factory delivered oversized, useless nails.
-
A politician’s success might be measured by approval ratings, driving behaviours to be popular rather than drive meaningful results.
-
School is designed to develop work or university-ready young adults, and success is measured with exams. As a result, much of school is often focused on doing better at exams without considering broader success factors for life and work.
-
Hospitals typically capture quality-of-care ratings based on patient recovery with at least one hospital being exposed for refusing patient admissions that might lower their ratings.
-
The familiar catch cry of ‘publish or perish’ in university circles has led to a phenomenon where academics are pressured to adjust study results with the ultimate aim of publication numbers over impactful research.
-
An individual who is wanting to be healthier might use weight loss as a measure, only to make unhealthy decisions that fast-track weight loss.
VARIATIONS – CAMPBELL’S LAW & THE COBRA EFFECT.
In a speech in 1975, Charles Goodhart jokingly coined Goodhart’s Law explaining “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” This concept was popularised and simplified by Keith Hoskins and Strathern (see Origins below for more).
In 2018, David Manheim and Scott Garrabrant from the Machine Intelligence Research Institute identified four types of Goodhart’s Law to help inform AI development, which we’ve summarised below:
|
Type of Goodhart’s Law |
Description |
Example |
|
Regressive |
Choosing a measurement that is a single proxy to your goal, when the goal actually has several causal factors behind it. |
Knowing that successful basketballers tend to be tall, you might build a basketball team by recruiting tall people. However, they might not possess other factors such as coordination, fitness, quick responses etc.. |
|
Extremal |
Where a proxy might indicate the goal under normal circumstances, but not in variable contexts. |
Sugar consumption helped define our ancient ancestors’ survival however today, with a different context, our excessive sugar consumption is often an indicator of an unhealthy diet. |
|
Causal |
Confusing Causation vs Correlation when selecting measurements. |
Data might indicate increasing ice cream sales when people wear sunglasses. However, providing more sunglasses won’t necessarily boost ice cream sales because they are both caused by a third factor, ie. sunny, hot days. |
|
Adversarial (Cobra Effect) |
Choosing a proxy measurement incentivises people to make the proxy the new goal. |
Adversarial aligns with the Cobra Effect, a term stemming from British-ruled India when, as the story goes, a local government body offered a bounty on cobras to reduce their population. Unfortunately, this led to people breeding more cobras for the bounty. |
In addition, Campbell’s Law represents an added perspective on the Adversarial approach in particular, by pointing out that the more a measure becomes valued, the more pressure there is to cheat or corrupt the data to deliver it. For example, Campbell explained “Achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.”
TECHNIQUES TO CHALLENGE GOODHART’S LAW.
Goodhart’s Law highlights challenges with measuring anything as a proxy for a goal, but it should not be mistaken for dismissing all attempts at measurements. Goodhart himself was not against statistical analysis, his criticism of the Thatcher government was that such statistics were defining monetary policy. So, before you give up on measuring, here are some techniques you might want to consider:
-
Pre-Mortems: When establishing a metric that matters, take time to consider downstream consequences using Inversion and Second Order Thinking. In simple terms, this involves exploring challenges that you should avoid and considering how people might change behaviour to focus on shifting the metric over reaching a goal.
-
Authentic metrics: an obvious improvement is to find proxies that more fully represent your actual goal or outcomes of your goal, so for example with the Cobra Effect described above, this might have been about rewarding a measure around fewer living cobras rather than dead ones. This is obviously problematic though, as measuring such elements will generally be more complex. The shift will not necessarily solve the adversarial challenge of being corrupted, for example measuring the number of living cobras might lead to fewer people reporting sightings to meet the metric.
-
Pairing indicators to create tension: this approach involves finding two counterbalanced metrics that can more accurately represent a healthy tension to move closer to a broader goal. For example, if you want to improve sales you might have a metric around the number of customer of conversations for a sales rep. This in turn might lead to fast but useless conversations to boost that metric. It can be paired with another metric such as NPS scores that attempt to reflect the quality in addition to the number of conversations (though see the In Practice section for challenges with the NPS).
-
Broaden success measures: taking the pairing approach to the next level you might consider broader metrics, both quantitative and qualitative to better establish progress against a goal. The Balanced Scorecard is one business-focused example here.
IN YOUR LATTICEWORK.
We’ve already discussed how you might combine Inversion and Second Order Thinking to consider the negative impacts of a seemingly useful measure. The Balanced Scorecard is one way of viewing a broader number of measures to consider business health. In addition, it might be worth considering Systems versus Goals to move away from goals and proxy metrics altogether or at least explore additional forms of Feeback Loops beyond traditional metrics.
-
Be wary of setting indicator measurements as targets.
Goodhart’s Law is a reminder that measures that become targets will distort behaviours in generally undesirable ways as people attempt to game the system. It doesn’t mean you should never set measures, but it is a warning to consider the downstream implications.
-
Seek more authentic and multiple metrics.
Rather than being focused on a single metric, consider using multiple measures that better represent your goal, and try to move those metrics as authentically close to your actual goal as possible.
Goodhart’s Law essentially points out the limitations of measures that are substituted for goals, so it is difficult to critique. Perhaps the best limitation in relation to Goodhart’s Law is the suggestion that you should give up on measures as a result of it. Instead, this ‘law’ presents a strong reminder to decouple rewards with your chosen metrics and to continually challenge the measures you are using to ensure that they do not drive undesirable behaviours.
Big Mac Index.
Given McDonald’s global coverage and the basic indicator of food pricing linked to inflation, for a time the Big Mac Index was used to test the level of a country’s inflation. This simple index was, as it sounds, literally charting the price of a Big Mac to chart the level of inflation. Though in the grips of rising inflation in the early 2000s, the Argentinian government persuaded Mcdonald's to keep Big Mac prices low so they could game the system.
Net Promoter Score.
This HBR article describes challenges with the broadly accepted NPS measurement in assessing customer satisfaction. It explains research that demonstrated that customers might be both detractors or advocates of a brand depending on context or product lines. Twitter was the best example of this, with many consumers having a ‘love-hate’ relationship with the platform that could not be captured by a single NPS score.
Others have noted that NPS scores can be shifted by brands desperate for better ratings simply by framing the question differently, e.g. verbally mentioning to a customer that ‘a score of 8 or above indicates that you’ve been satisfied by my service’.
d
Goodhart’s Law is named after British economist Charles Goodhart who, in 1975, jokingly coined the term, stating that “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” His original formulation was part of his critique against the Thatcher government, which he argued was implementing monetary policy based on narrow statistical targets.
Our modern understanding of the law can be attributed to Keith Hoskins who, in 1996, wrote: “'Goodhart's Law' – That every measure which becomes a target becomes a bad measure – is inexorable, if ruefully, becoming recognized as one of the overriding laws of our times.” Anthropologist Marilyn Strathern responded to Hoskin’s paper in 1997 and captured the law in the popular phrasing it is most commonly known today.
Our overview references the work of Manheim and Garrabrant’s Categorizing Variants of Goodhart’s Law which can be seen here. A summary of the paper can be found on Garrabrant’s blog post here.
My Notes
Oops, That’s Members’ Only!
Fortunately, it only costs US$5/month to Join ModelThinkers and access everything so that you can rapidly discover, learn, and apply the world’s most powerful ideas.
ModelThinkers membership at a glance:
“Yeah, we hate pop ups too. But we wanted to let you know that, with ModelThinkers, we’re making it easier for you to adapt, innovate and create value. We hope you’ll join us and the growing community of ModelThinkers today.”
You Might Also Like:
- Actionable summaries of the world's most powerful ideas.
Subscribe now and download this practical guide to transform your thinking, plus recieve one short, powerful and actionable idea in your inbox each month.