What COVID-19 antibody tests can tell us, and what they can’t

As the United States and much of the world move toward relaxing shelter-in-place restrictions to let people move about more freely, public health experts hope to rely on antibody tests to determine who has been infected with the COVID-19 virus and may be immune — at least temporarily — and who is still susceptible.

A University of California, Berkeley, and UC San Francisco project to evaluate some of the more than 120 available antibody test kits — only a handful of which have received emergency use authorization from the Food and Drug Administration — should provide the test performance data these doctors and public health officials need to decide which tests to employ and to understand how reliable the results are.

In head-to-head comparisons of a dozen tests, the researchers already have found that many of the tests performed reasonably well, especially two weeks or more after infection, when levels of antibodies in the blood begin to peak. But many of the test kits have false positive rates that may exceed the proportion of people who have been infected in some communities. That means that a large proportion of those testing positive on an antibody test may not actually have had COVID-19.

Some government officials have suggested providing those who test positive with a get-out-of-jail-free card, an “immunity passport” to a normal life. However, additional critical information is still required before assuming that antibody tests can safely predict protection from future infections, the researchers cautioned.

“It’s the Wild West right now. These tests are widely available, and many people are buying and deploying them, but I realized that they had not been systematically validated, and we needed to figure out which ones would really work,” said Patrick Hsu, an assistant professor of bioengineering at UC Berkeley and an investigator at the Innovative Genomics Institute (IGI), a joint research collaboration between the two campuses that is focused on CRISPR. “This is a huge, unmet need for public health.”

Hsu is leading the effort with Alex Marson, an associate professor of microbiology and immunology at UCSF and IGI’s scientific director for biomedicine, Caryn Bern, a professor of epidemiology and biostatistics at UCSF, and Jeffrey Whitman, a clinical fellow in pathology at UCSF and a resident in laboratory medicine.

They have posted their first results online at https://covidtestingproject.org/, in advance of peer review and submission to a journal, and will continue to update that website so that state and federal policymakers have the information they need before purchasing serology tests. The team cautioned that, as a preliminary report of work that has not been certified by peer review, it should not be relied upon to guide clinical practice or health-related behavior and should not be reported in news media as established information.

Antibody tests complement PCR diagnostics

Current diagnostic tests, such as the standard RT-PCR (reverse transcriptase-polymerase chain reaction) test conducted on samples obtained from nasopharyngeal swabs, can tell doctors if someone is currently infected, but antibody tests might be able identify people who have been exposed to the virus even weeks after their initial infections. Antibody tests could be particularly useful for identifying those who were infected, but never showed symptoms. Some controversial studies have suggested that the proportion of such cases could be as high as one in four.

When infected by a virus like SARS-CoV-2, the cause of COVID-19, the body initially produces antibodies known as IgM (immunoglobulin-M), in an attempt to neutralize the virus. Later, as the body’s adaptive immune system revs up, IgM levels go down, and the body ramps up production of IgG, which more specifically targets the viral invader.

Antibody tests, also called serology tests because they are conducted on blood samples, such as from a finger prick, can assess levels of both IgM and IgG, and the relative levels could indicate whether a person is in the early or late stages of infection.

As such, antibody tests can complement the information from PCR tests, since even these relatively accurate tests can give false negatives. PCR tests for coronavirus generally exhibit lower sensitivity if performed several days to a week after symptom onset, probably because of decreasing levels of the virus in the upper respiratory, tract where samples for testing are commonly taken.

Antibody tests may eventually provide clues to how long immunity lasts and what levels of antibodies are truly protective against subsequent SARS-CoV-2 infections. It’s unclear, for example, if infection with SARS-CoV-2 produces a long-lasting immunity.

The UC Berkeley/UCSF team has so far evaluated 10 point-of-care tests — tests much like home pregnancy or HIV tests, which are called, in general, lateral flow assays — and two different set of tests based on a common laboratory antibody detection method called ELISA (enzyme-linked immunosorbent assay).

Each of the dozen was tested against roughly 300 blood samples. Of these, 108 were obtained before July 2018, so, presumably, from people who had not contracted COVID-19. Most of the remainder came from COVID-19 patients seen at Zuckerberg San Francisco General Hospital and Trauma Center (ZSFG) or UCSF Medical Center. About 130 samples were from individuals who had tested positive for COVID-19 by PCR testing, and around 50 were from people who had been tested for other viruses.

“We have carefully curated the specimens so that we can systematically study how these different antibody tests perform at different times since symptom onset, and across many samples, making our study one of the most comprehensive to date,” Hsu said.

Nevertheless, the team is hampered by the lack of a definitive antibody test with which to compare the many new test kits on the market.

“One of the cornerstones of lab medicine is that a new test is compared to a definitive reference or gold standard,” Marson said. “We do not have a gold standard yet for COVID-19 serology testing, so we are amassing data on a standardized set of blood samples and really looking at how each of these tests performs in relationship to all the others.”

Sensitivity vs. specificity

The COVID-19 patient samples represented blood taken at various stages of illness, starting about three days after initial onset of symptoms. Patients ranged in age from 22 to over 90 and were primarily (69%) of Hispanic/Latinx ethnicity — the demographic largely served by ZSFG and one of the segments of San Francisco’s population hit hardest by COVID-19.

Each test was assessed for how likely it was to detect antibodies in the blood of a coronavirus-positive patient and also for specificity, that is, how good the test was at distinguishing a person who was uninfected. A high specificity means a low false positive rate.

Of the 10 point-of-care tests, “there are multiple tests that have specificities greater than 95%. So, there is some reason for guarded optimism,” Marson said. “Although, it is important to point out that if these infections are rare in a population, a false positive rate of 5% could cloud the picture of the information coming in.”

“Several of our tests had specificities over 98%, which is critical for reopening society,” Hsu said.

The extent to which positive results by serology may reflect a protective immune response will require further study, Hsu emphasized. “More research is needed to understand if antibody assays can be used as predictors of protection against re-infection and to prioritize return to work,” he said.

Hsu and Marson noted that, while point-of-care tests are meant to be yes/no tests — either you have antibodies or you don’t — they actually display positive results within a range that can be helpful in judging how certain the results may be. Doctors can set a high bar — a darker band on the dipstick-like tests — to be more certain of negative results, but it would come at the expense of losing sensitivity to small antibody levels.

“Many of the false positives were associated with fainter bands,” Marson noted.

“Accurate use of these tests will depend on adequate training of test readers,” Hsu added. The researchers are now working on methods to standardize rapid serology test interpretation that could eventually be used with a cellphone camera.

The UCSF/Berkeley team also worked with researchers at Massachusetts General Hospital who had independently assessed three antibody kits. One of those kits overlapped with the Bay Area team’s set of kits and confirmed its assessment of it.

Community effort

Hsu, a geneticist, works on CRISPR systems that target RNA, so when SARS-CoV-2 came along — an RNA virus — he started to think about ways to use CRISPR enzymes to create a rapid, point-of-care diagnostic for the virus. While he is still working on that project, he learned in March of troublesome issues with antibody tests, and he and his lab colleagues began to acquire some of the tests available — most of them from China and South Korea — in hopes of evaluating them.

In one of the weekly Zoom meetings within IGI, he learned of Marson’s interest in the same issue, and they teamed up to acquire and evaluate more antibody test kits. They recruited Whitman and Bern, who have experience evaluating antibody test kits for other diseases. Whitman was able to acquire patient blood samples from ZSFG and UCSF and use lab equipment in ZSFG’s Department of Experimental Medicine.

With an army of dedicated researchers from UCSF and UC Berkeley, the team has been working around the clock to acquire new test kits, run the assays and evaluate the results.

They continue to expand the number of tests they are evaluating. They are also obtaining new blood samples from patients who have recovered from COVID-19 to look at antibody levels long after recovery in order to determine how long antibodies stick around and how that correlates with severity of the illness and subsequent immunity.

“This is a huge, huge community effort,” Hsu said. “A lot of people really came together. One of the things I think is cool about this study is how many people repurposed themselves from what we normally do to respond to this pandemic. Personally, I find it very inspiring.”

The research is supported by grants from the Chan Zuckerberg Biohub, Anthem Inc. and other local donors.