Richard J. C. Brown, Head of Metrology and Jan-Theodoor Janssen, Chief Scientist from the National Physical Laboratory, Teddington, explore why the system doesn’t work if everyone is measuring COVID-related deaths differently and how a standardised international definition could bolster public trust
As countries are learning to live with the SARS-CoV-2 pandemic in the long term, attention turns to comparing the effectiveness of countries’ strategies in supressing the spread and effects of the virus. One key summary statistic is the “number of SARS-CoV-2 associated deaths”. This statistic seems to be emerging as more important than statistics related to positive tests – these can depend on the number of tests performed and whether those tests are targeted as in areas with high infection rates, or are taken randomly as in the Office for National Statistics infection survey (1).
Reporting deaths
Reporting the number of people who have died clearly and consistently is challenging and needs careful handling to maintain public trust in the science presented. The difficulty stems not from our ability to count deaths because they are too numerous, nor because we can only sample a proportion of the target population – deaths caused by SARS-CoV-2 may be counted in their entirety – but from uncertainty in what is counted. Put simply, we need to describe in enough detail how we are doing the counting, what is included and what is not, otherwise it is unclear, or uncertain, what we mean by our measurement result. For instance, does this figure include only deaths in hospital, only those accompanied with a positive diagnostic test, those where SARS-CoV-2 is mentioned on the death certificate, death before a certain time cut-off, or all excess deaths over and above the long-term average? This makes a big difference.
UK COVID deaths
Currently, Public Health England publish three figures (2): ‘death within 28 [of a positive test]’, ‘death within 60 days [of a positive test], or Covid-19 on death certificate’, and ‘no time restriction imposed [after a positive test]’. On the 8th September these numbers were 9, 19 and 53, respectively. These are large differences and are enough to alter people’s views, and perhaps even policy decisions, about the pandemic. In July the Government reviewed how the headline deaths figure was calculated in England and in August, as a result of this review, moved towards using the 28-day cut-off for reporting deaths. This had the additional benefit of unifying the method of reporting across the UK (3).
Whilst the UK may have unified its approach recently, different countries still use a range of approaches for calculating and reporting data related to the pandemic. There are many options for reporting deaths – all generally credible metrics. What is being counted is usually well documented within individual countries (4), even if opinion on what should be counted is not unified. Just as a unified approach in the UK was important, so international agreement on a standardised approach is urgently needed, otherwise we risk obfuscating identification of the most effective virus suppression measures and confusing communication of the data to the public – attempting to explain that not every death is equal is an unedifying and ultimately futile task.
Measurement science
We have recently argued (5) that widespread adoption of a unified approach to applying sound, thoughtful measurement science in counting deaths is essential to retain public trust in the numbers presented, and ensure the global comparability and reproducibility needed to learn the lessons of the SARS-CoV-2 pandemic as quickly as possible. Indeed, this approach is also more widely applicable to other SARS-CoV-2 problems—for instance, agreed standardised methods to measure false positive or false negative rates of antigen and antibody testing (6), especially when the marketplace is increasingly competitive.
The measurement science community – such as the international network of national metrology institutes (NMIs) – already ensured the standardisation and comparability of the measurements we rely on for our ‘normal’ lives. They now must be at the forefront of leading efforts to do the same for the data produced to characterise the pandemic in our ‘new normal’ lives
Resources
(1) Coronavirus (COVID-19) Infection Survey pilot Statistical bulletins, https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/previousReleases
(2) PHE data series on deaths in people with COVID-19: technical summary, https://www.gov.uk/government/publications/phe-data-series-on-deaths-in-people-with-covid-19-technical-summary
(3) Coronavirus: England death count review reduces UK toll by 5,000, https://www.bbc.co.uk/news/health-53722711
(4) Daily new confirmed COVD-19 deaths per million people, https://ourworldindata.org/grapher/daily-covid-deaths-per-million-7-day-average
(5) Brown, R.J.C., Janssen, J.T. Counting correctly. Accred Qual Assur (2020). https://doi.org/10.1007/s00769-020-01444-6
(6) Diagnostic tests for covid-19—improving accuracy and global harmonisation. https://blogs.bmj.com/bmj/2020/05/06/diagnostic-tests-for-covid-19-improving-accuracy-and-global-harmonisation
