Antibody tests were meant to be a game-changer. What went wrong?

Gil Cohen-Magen / Getty Images

After falling ill in early March with mild cold-like symptoms, Stephen Graves couldn’t shake the niggling feeling that he may have contracted Covid-19. Like all UK residents who displayed coronavirus symptoms in early March, Graves wasn’t eligible for testing, which at the time was mainly limited to those in hospitals.

On May 25 he ordered an antibody test from a private London clinic that would analyse a blood sample – taken from the tip of his finger – and tell him whether he had been infected with the virus that causes Covid-19, Sars-Cov-2. Four days earlier, the health secretary Matt Hancock had said at the daily Downing Street press conference that the government was considering “systems of certification” that might give people with antibodies more freedom to return to work. More than that, Hancock had already floated the idea weeks earlier, and since then the internet had been abuzz with speculation about whether such a scheme could, or should, ever be implemented.

For Graves, who returned his finger-prick blood sample by post on June 13, the motivation was even more basic: curiosity. “If you had a cold during that early March period you’re sitting there going, ‘Well, did I have it or didn’t I?’ So that was the main reason I just wanted to find out,” says Graves.

As it turned out, Graves is still in the dark about his result. Between ordering his test and returning it, the Medicines and Healthcare products Regulatory Agency (MHRA) asked companies to stop selling antibody tests using finger-prick blood samples as the tests had only been validated using blood taken directly from veins. Graves’ test result, which came back negative, could not be relied upon.

For the many UK residents who ordered finger-prick antibody tests from Superdrug, Lloyds Pharmacy, Babylon Health and other private retailers, the experience was little more than an annoying inconvenience. But it was not the first time things had gone awry with the UK’s handling of antibody tests. In April, prime minister Boris Johnson had proclaimed a forthcoming shipment of Chinese antibody home test kits “as simple as a pregnancy test” (albeit one using blood, not urine) and having the potential “to be a total game changer.” In fact, the two million hastily-ordered tests never worked and officials were left searching for a way to get their money back.

Deployed correctly, antibody tests are a powerful tool. They can tell us which areas have been hit hardest by the pandemic – they are how we know that, as of the end of April, about 17.5 per cent of Londoners had been infected with Covid-19, with considerably lower infection rates elsewhere. If we can establish the link between Covid-19 infection and immunity, antibody tests may one day be able to tell us who is less vulnerable to the disease. But in the UK, antibody testing has been plagued by the devils of over-promise and under-delivery. First dangled as a silver-bullet the antibody tests – like the NHS Covid-19 contact tracing app – risk becoming another would-be solution discarded by the wayside.

Things started to go wrong with the UK’s approach to antibody testing almost as soon as the epidemic got underway, says Jon Deeks, professor of biostatistics at the University of Birmingham, who is currently analysing data about the accuracy of Covid-19 antibody tests used all over the world. As with so much of the Covid-19 response, it all comes down to testing – or the lack of it. It wasn’t until May 19 that the UK government opened up the testing criteria so that anyone aged over five with coronavirus symptoms could have a test. Before then swab tests – which test for the presence of the virus but can’t tell if you’ve had it in the past – were restricted first to those in medical settings, and then to key workers, their families and other groups. “We have a lot of consumer demand [for antibody tests] in the UK. And a large amount of it is driven by the fact that we didn’t have the capacity to test people when they had the disease,” Deeks says, adding it would have been much better to test people when they were ill to see if they had Covid-19.

As the number of UK Covid-19 cases started to spiral out of control in early March, officials were struggling to meet 5,000 coronavirus tests a day. At the same time, Germany was producing nearly 100,000 tests per week. While this had serious implications for contact tracing and measuring the extent of the disease spread, the limited testing created another problem: a vast group of people who suspected they’d had the virus, but couldn’t prove it.

For certain diseases, knowing whether you’ve had it or not doesn’t matter an awful lot. People don’t usually clamour to know which specific virus caused them to have the sniffles or how many times they’ve had a tummy bug in the last three years. What difference does it make?

But at the same time as tens of thousands of people were left unable to find out if their symptoms were caused by Covid-19, the UK government was starting to suggest that whether you had the disease or not in the past may matter a great deal. On April 2, Hancock said that the government was “looking at an immunity certificate,” adding that “people who have had the disease have got the antibodies and then have immunity can show that and therefore get back as much as possible to normal life”.

The idea of combining immunity passports and coronavirus appears to have originated in the pages of the German newspaper Der Spiegel which on March 27 published an interview with researchers at Helmholtz Centre for Infection Research in Braunschweig who were planning a large antibody study in Germany. Gerard Krause, the epidemiologist leading the research, said that the information could be used to issue “something similar to a vaccination certificate” that would give people exemptions on limits to their activities.

By the time the article had made its way into the UK press, it was widely reported that the German government was considering the immunity passport scheme – although The Guardian noted on March 30 that the German government had made no comment on the proposal. Three days later, however, Hancock appeared to suggest that at least one national government really was considering the idea. Although he warned that the science around immunity passports was “too early”, the health secretary had raised the idea before an audience of millions that knowing your Covid-19 status could one day prove very important indeed. Suddenly, the thought was planted in the minds of millions of Britons that knowing your Covid-19 status might be very important indeed.

How antibody tests are talked about to the public is absolutely crucial, says Joe Fitchett, medical director at the UK-based medical device firm Mologic, which is in the late stages of developing and validating multiple antibody tests of its own. “Sometimes the use case might benefit the population and surveillance, but it might not necessarily make a difference at an individual level,” he says. To epidemiologists and public health officials, antibody tests are a powerful tool for tracking the spread of infections, but to an individual they may do nothing more than a way to satisfy a lingering curiosity.

If antibody tests were to be useful on an individual level, we’d need them to tell us something about whether someone who has been infected with Covid-19 is immune to the disease afterwards. And, right now, there isn’t enough evidence to give any clear answers. Fitchett says it’s still too early to tell whether people who have Covid-19 antibodies cannot contract the disease again. “This is a brand new virus that to our knowledge emerged at the end of last year. And so, by definition, we need to follow it over time.”

There are promising signs that being infected with Sars-Cov-2 does provoke an immune memory that may help the body fight the virus if exposed to it for a second time. Experiments in macaque monkeys found that the animals were resistant to catching Covid-19 for a second time five weeks after being initially infected. Many immunologists are cautiously hopeful that the body can mount at least some kind of immune response to the disease.

Much of what we think we know about immunity to Covid-19 is based on our knowledge of other human coronaviruses, writes the Harvard epidemiologist Marc Lipsitch in the New York Times: “After being infected with SARS-CoV-2, most individuals will have an immune response, some better than others. That response, it may be assumed, will offer some protection over the medium term.” But for the most part, immunity to Covid-19 still remains a mystery. We don’t have reliable answers for some of the most important questions about the disease. How does immunity vary from person to person? Do we become less immune as time goes on? Is the level of immunity linked to infection severity? Are older people less likely to be able to mount an immune response?

To put it bluntly, we know vanishingly little about immunity to Covid-19. And individuals won’t find any answers about immunity in their antibody test results. “It’s hopeful that antibodies are evidence that you are immune, but we don’t know how long that immunity lasts – if it’s there – and not all antibodies are what are called neutralising antibodies which are the ones which sort of eat up the virus,” says Deeks.

The British government, however, has left little room for doubt. In late March, Johnson promised that an upcoming shipment of antibody tests would be a “game changer”. It was these tests – a shipment of two million from China – that he promised would be as simple as taking a pregnancy test. As The New York Times reported, the UK was asked to pay at least £16 million for the tests from two Chinese companies up front. An analysis by a laboratory at the University of Oxford, however, soon revealed that the tests were so inaccurate that they were of no use at all.

After the disappointment of the Chinese tests, it would be another month before the government found another set of antibody tests to get excited about. On May 3 the Swiss healthcare firm Roche announced that the US Food and Drugs Administration had issued an emergency use authorisation for its antibody test. The press release announcing the news contained some impressive statistics suggesting that the test was 100 per cent sensitive, which would mean that it is always able to identify every single person who did have Covid-19 antibodies and give no false negatives.

By the time they reach the popular press, statistics about test accuracy are often garbled beyond recognition, as statistician David Spiegelhalter warns in The Guardian. Very quickly, the Roche test was reported as being “100 per cent accurate.” Less than two weeks later, when the UK’s national coordinator of Covid-19 testing, John Newton, announced the results of Public Health England’s (PHE) analysis of the Roche test, the same “100 per cent accurate” claim led dozens of news reports about the new test. On May 15, the UK government closed a contract with Roche to buy £13.46 million worth of the tests until November 2020, with the possibility of extending the contract for another six months.

But the results of PHE’s evaluation of the Roche test differed considerably from the firm’s press release. While Roche reported that its test never gave a false negative (100 per cent sensitivity), PHE’s evaluation found that it actually was only 83.9 per cent sensitive, rising to 86.7 per cent if the antibody test was conducted more than 21 days after the onset of Covid-19 symptoms. This means that sometimes the Roche test returns a negative result for antibodies even when an individual actually has been infected with Covid-19 in the past. The reason for the difference between the two evaluations, a spokesperson for Roche says, is because the company evaluated its test’s performance with blood taken two weeks after the individual had a positive Covid-19 swab test, while PHE’s baseline was two weeks after the first appearance of Covid-19 symptoms.

Arguably, the more important statistic for this kind of test is its specificity, which is how often they give false positive results – telling someone they have Covid-19 antibodies when they really don’t. Sending someone back to work because you think they have antibodies against a virus would, obviously, be a very bad idea without a test you could trust to tell you that information (Tom Chivers has written about the perils of using tests in this way in UnHerd). Here, the Roche test performed even better than the company’s press release claimed, reaching 100 per cent specificity in the PHE evaluation, and it is this specificity measure that Newton was referring to and later was misreported as a “100 per cent accuracy” claim.

Neither the Roche test nor one produced by Abbott, that the UK government has also bought at scale, meet the MHRA’s challenging targets of both 98 per cent sensitivity and 98 per cent specificity. This doesn’t mean that either test is useless – far from it – but it highlights the compromises that have to be made as governments scramble to get their hands on antibody tests as quickly as possible. “As is standard practice, contracts were only placed when Public Health England’s evaluation of each test was confirmed,” a DHSC spokesperson says, noting that the government also has contracts for antibody tests from Ortho Clinical and Siemens.

In mid-May, with the government’s self-imposed target of 200,000 coronavirus tests (later changed to testing capacity) per day looming, this emphasis on speed may have been felt particularly acutely. The government was only able to meet this target at the end of May after including capacity for 40,000 antibody tests. At the end of May, NHS England instructed hospitals to begin mass testing of staff and patients, helping push the government over its target for the month.

Fitchett says that this focus on rapid, automated testing may have meant the UK missed the opportunity to bring out antibody tests at an earlier stage. Mologic sent its laboratory antibody tests for validation by PHE on April 21, but that the validation was postponed while PHE prioritised the validation of antibody tests intended to work with automated platforms that can analyse up to 300 tests every hour. “They could have ramped up testing of NHS staff and patients in April whilst awaiting validation of the automated platforms,” Fitchett says. “What it points to in my mind is a focus on a more centralised testing strategy rather than a more distributed approach seen elsewhere.”

It was these same tests that were quickly snapped up by high street pharmacies and private clinics and sold for use with finger-prick blood samples, before the MHRA said they could not be sold as the tests had not been validated for use with finger-prick blood samples. Despite this, some pharmacies still list the finger-prick test online, saying they are currently out of stock. “We’ve got a regulatory process which fails us. It doesn’t protect the public and it doesn’t help the government make wise decisions,” says Deeks.

Fitchett says that the bumpy road that antibody tests have followed so far is partly due to the pressure of unrealistic expectations and lack of communication about how and where the tests could be most effectively put to use. In a pandemic, hype cycles run in fast-forward. “I think the pendulum is swinging, it’s swung towards over promise. It’s now swinging towards this perception that they’re completely useless. And I hope it will just level like it will with any diagnostic test in medicine, which is it has its uses under certain circumstances, depending on the question you want to answer,” Fitchett says.

Kristal Ireland, who bought an antibody test from Babylon Health in mid-May, was looking for a little certainty about the future. “I just thought for me personally it would eliminate some of that fear of whether I did have it and whether I was going to get ill,” she says. Having been through lockdown with her young daughter, £69 seemed a small fee to pay for a little peace of mind. After milking the blood from three of her fingers, Ireland ended up receiving a refund instead of a test result.

But even if she did test positive for Covid-19 antibodies, Ireland said it wouldn’t have changed her behaviour. Even though NHS England is delivering widespread antibody testing for its staff, it has emphasised that the tests don’t tell us anything about whether someone can be re-infected with the virus, whether they can pass it on to others, or if they have protective immunity. Perhaps the biggest problem at the heart of the antibody test rollout is that they are not capable of delivering the information that many people desperately want to know.

After months of being told that antibody tests would be a gamechanger, and the key to unlocking, if not the country, then at least the lives of some individuals, we are faced with the reckoning that right now, such tests are of relatively little individual benefit. And while the government talked-up the individual benefits while overlooking their power as disease-tracking tools, people may be left wondering what all the fuss was about. “If there’s no decision that anyone should be making based on these tests, then why do it?” says Deeks.

Matt Reynolds is WIRED’s science editor. He tweets from @mattsreynolds1

More great stories from WIRED

☢️ Nine years on, Fukushima’s mental health fallout lingers

? Google got rich from your data. DuckDuckGo is fighting back

? Which face mask should you buy?

? Follow WIRED on Twitter, Instagram, Facebook and LinkedIn