One of the hills I will die for, is that indicators are of two types – lagging (they estimate what happened) and leading (they estimate what will happen) – and that following the former causes us to react only after the worst came.
Let me explain.
Note, if you already know what lagging and leading indicators are, keep reading nevertheless – you won’t be disappointed. This is not about theory, but practice.
Number of deaths is a lagging indicator. It tells you how bad the epidemic has been, but it carries little information regarding the future. For example, according to official numbers, Italy and China have a comparable amounts of deaths – however, the pandemic is out of control in the former but not in the latter. This means that number of deaths alone has little predictive power on how things will work out [1].
Instead, the percentage of people wearing a face mask is a leading indicator. It has some predictive power on how the epidemic will look like in a few weeks – the more people were the mask, the less contagions there will be.
Now, the problem with lagging indicators
When lagging indicators are used, errors surface only when it’s too late. If you wait to begin a lockdown until you have a few hundreds deaths, a lagging indicator, then (1) you will begin it too late and (2) by definition, you will have a few hundred non-preventable deaths.
Conversely, using leading indicators highlight problems before their consequences materialize. If you begin reacting to a possible pandemic whenever you see a spike in hospitalizations regardless of cause, you will react (1) when the measure have a shot at preventing an outbreak in your country and (2) early enough save the first few hundred deaths. This is what Taiwan did: it read WHO’s reports and began screening travelers from Wuhan as early as the 5th of January.
Of course, other factors matter too: competence of governments, operational effectiveness and population preparedness, for example. However, a proper use of leading indicators allow you to spot bottlenecks in them before you need them to work properly, whereas relaying on lagging indicators is basically a form of crossing fingers and hoping that governments are competent, operations are effective and the population is prepared.
To summarize the last few paragraphs: lagging indicators are fragile to implementation quality (errors show up when it's too late) whereas leading indicators are antifragile to implementation quality (poor execution visibly worsens them, so errors are spotted quickly and corrected immediately).
An example: if you focus on number of people having social contact without PPE (a leading indicator), chances are your action will have a positive long-term effect, as you iterate proactively. If, instead, during a lockdown you focus on the number of deaths (a lagging indicator), if the lockdown is not working properly you will discover it too late, after a couple of weeks of subsidies and wasted economy.
The implication: any lockdown whose implementation is adapted based on lagging indicators will take too long to resolve, and will cost too much. In order to be quick and effective, a lockdown has to be adapted based on leading indicators.
Where to look for
In one of my previous jobs, I was consulting manufacturing companies on behavioral safety – how to make sure that their workers would not injure themselves. In behavioral safety, there is this concept called Pyramid of Risk.
The pyramid is based on the realization that for each death on the workplace there are many reported injuries; for each reported injury, many minor ones; for each minor injury, many near misses (e.g., a brick falling from the roof but hitting no one) and for each near miss, many unsafe behaviors (e.g., not wearing Personal Protective Equipment, PPE).
Two key concepts:
If you use metrics from the top of the pyramid, you are waiting for deaths to happen before you take action, whereas if you use metrics from the base, you can act before people die.
Metrics from the top of the pyramid have a small sample and are thus less reliable; metrics from the basis have a larger sample and are more reliable.
Here is one version of how the Pyramid of Risk could look like for the coronavirus:
Cargo cults
“Cargo cult” is an expression that relates the post-WW2 era, to Pacific island natives who saw US personnel signaling cargos how to land on their island and, after they were gone, kept repeating for decades the same gestures as a ritual to get planes to land. Nowadays, “cargo cult” is used to refer to any group of people which repeats a series of actions by imitation, without an understanding of the cause-effect process.
Observing the reaction of many governments to the pandemic, I see a lot of cargo cults. Western governments are copying some of the things that they saw the Chinese do (for example, the lockdown), without really understanding how and why it worked. The result: a costly mess which might not even work.
Cargo cults are the celebration of lagging indicators. They are defeated by embracing leading ones.
Practical implications
Regarding the implementation and optimization of lockdowns:
Not all lockdowns are equal. In China, the lockdown was extremely strict, well enforced and was paired with door-to-door screening and isolation of the sick in dedicated structures. Italian lockdown still allow people to go out to the grocery store and is weakly enforced.
To know whether a lockdown in a given country is working, we need to measure indicators.
If we observe lagging indicators and the lockdown is not working as designed, we will discover it too late.
Therefore, we should focus on observing leading indicators, so that we can iterate faster on its implementation details.
Some leading indicators I would focus on:
People wearing PPEs. If people go out OR if they stay in the same apartment with someone infected, they might catch the virus. Wearing PPEs, including some makeshift ones under the right conditions (link), is a necessary condition to prevent contagion during a lockdown (unless it has been initiated so early that the chances of being infected are very low).
Door-to-door screening is the only way that we can have a short lockdown. Without it, the minimum duration is five weeks, as proposed by Nicholas Nassim Taleb and Yaneer Bar-Yam: two weeks for the infected to become infectious, two weeks for their housemates to become symptomatic, and one more week as a safety cushion. Otherwise, we might not catch all cases, unless door-to-door screening is implemented.
Testing capacity, unless it is ramped-up, we will have long lockdowns and possible second waves.
Of course, this is not an exhaustive list. But I hope that the main point is passed: if we only look at lagging indicators, we will react slowly. Focusing on the right leading indicators is the only way we can have a chance at defeating the pandemic before it cripples us.
Leading and lagging indicators are better explained in my book “Best Practices for Operational Excellence” (paperback and ebook).
Sign-up to this newsletter clicking on the button below:
Notes:
[1]: not even a curve of deaths has much predictive power. When will the next inflection point be? What were the quality and comprehensiveness of testing?
How to know if a lockdown is working
Hi Luca, I really enjoy the newsletter. On the topic of types of indicators is any indicator inherently 'leading' or 'lagging'? For instance, could number of people wearing PPE still be lagging behind some other metric? Like number of people out in public. (If we could calculate that and know that it leads to a portion of people not wearing PPE in public. )