One of Peter Drucker’s most known quotes is “If you can’t measure it, you can’t improve it.” It’s hard to deny that measurement and gathering data are critical factors to success, which is why Goodhart’s Law can be a harsh reality check.  In the words of anthropologist Marilyn Strathern, Goodhart’s Law tells us that ‘when a measure becomes a target, it ceases to be a good measure.’ GOODHART’S LAW IS EVERYWHERE. Goodhart’s Law can be seen anywhere where people want to measure progress to reach a goal, for example:  A Soviet Union nail factory was famously said to have been given a target related to the number of nails and proceeded to create miniature, useless nails. When the target was shifted to weight, the factory delivered oversized, useless nails.  A politician’s success might be measured by approval ratings, driving behaviours to be popular rather than drive meaningful results.  School is designed to develop work or university-ready young adults, and success is measured with exams. As a result, much of school is often focused on doing better at exams without considering broader success factors for life and work.  Hospitals typically capture quality-of-care ratings based on patient recovery with at least one hospital being exposed for refusing patient admissions that might lower their ratings.  The familiar catch cry of ‘publish or perish’ in university circles has led to a phenomenon where academics are pressured to adjust study results with the ultimate aim of publication numbers over impactful research.  An individual who is wanting to be healthier might use weight loss as a measure, only to make unhealthy decisions that fast-track weight loss. VARIATIONS – CAMPBELL’S LAW & THE COBRA EFFECT. In a speech in 1975, Charles Goodhart jokingly coined Goodhart’s Law explaining “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” This concept was popularised and simplified by Keith Hoskins and Strathern (see Origins below for more).   In 2018, David Manheim and Scott Garrabrant from the Machine Intelligence Research Institute identified four types of Goodhart’s Law to help inform AI development, which we’ve summarised below:  Type of Goodhart’s Law Description Example Regressive Choosing a measurement that is a single proxy to your goal, when the goal actually has several causal factors behind it. Knowing that successful basketballers tend to be tall, you might build a basketball team by recruiting tall people. However, they might not possess other factors such as coordination, fitness, quick responses etc.. Extremal Where a proxy might indicate the goal under normal circumstances, but not in variable contexts. Sugar consumption helped define our ancient ancestors’ survival however today, with a different context, our excessive sugar consumption is often an indicator of an unhealthy diet.   Causal Confusing Causation vs Correlation when selecting measurements. Data might indicate increasing ice cream sales when people wear sunglasses. However, providing more sunglasses won’t necessarily boost ice cream sales because they are both caused by a third factor, ie. sunny, hot days. Adversarial (Cobra Effect) Choosing a proxy measurement incentivises people to make the proxy the new goal.  Adversarial aligns with the Cobra Effect, a term stemming from British-ruled India when, as the story goes, a local government body offered a bounty on cobras to reduce their population. Unfortunately, this led to people breeding more cobras for the bounty.    In addition, Campbell’s Law represents an added perspective on the Adversarial approach in particular, by pointing out that the more a measure becomes valued, the more pressure there is to cheat or corrupt the data to deliver it. For example, Campbell explained “Achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.” TECHNIQUES TO CHALLENGE GOODHART’S LAW. Goodhart’s Law highlights challenges with measuring anything as a proxy for a goal, but it should not be mistaken for dismissing all attempts at measurements. Goodhart himself was not against statistical analysis, his criticism of the Thatcher government was that such statistics were defining monetary policy. So, before you give up on measuring, here are some techniques you might want to consider:  Pre-Mortems: When establishing a metric that matters, take time to consider downstream consequences using Inversion and Second Order Thinking. In simple terms, this involves exploring challenges that you should avoid and considering how people might change behaviour to focus on shifting the metric over reaching a goal.  Authentic metrics: an obvious improvement is to find proxies that more fully represent your actual goal or outcomes of your goal, so for example with the Cobra Effect described above, this might have been about rewarding a measure around fewer living cobras rather than dead ones. This is obviously problematic though, as measuring such elements will generally be more complex. The shift will not necessarily solve the adversarial challenge of being corrupted, for example measuring the number of living cobras might lead to fewer people reporting sightings to meet the metric.  Pairing indicators to create tension: this approach involves finding two counterbalanced metrics that can more accurately represent a healthy tension to move closer to a broader goal. For example, if you want to improve sales you might have a metric around the number of customer of conversations for a sales rep. This in turn might lead to fast but useless conversations to boost that metric. It can be paired with another metric such as NPS scores that attempt to reflect the quality in addition to the number of conversations (though see the In Practice section for challenges with the NPS).  Broaden success measures: taking the pairing approach to the next level you might consider broader metrics, both quantitative and qualitative to better establish progress against a goal. The Balanced Scorecard is one business-focused example here.  IN YOUR LATTICEWORK. We’ve already discussed how you might combine Inversion and Second Order Thinking to consider the negative impacts of a seemingly useful measure. The Balanced Scorecard is one way of viewing a broader number of measures to consider business health. In addition, it might be worth considering Systems versus Goals to move away from goals and proxy metrics altogether or at least explore additional forms of Feeback Loops beyond traditional metrics.