A Comparison of Five SARS-CoV-2 Molecular Assays With Clinical Correlations

Gary W. Procop, MD, MS; Jay E. Brock, PhD; Edmunds Z. Reineks, MD; Nabin K. Shrestha, MD, MPH; Ryan Demkowicz, MD; Eleanor Cook, MD; Emad Ababneh, MD; Susan M. Harrington, PhD


Am J Clin Pathol. 2021;155(1):69-78. 

In This Article


All five SARS-CoV-2 nucleic acid amplification tests were performed on each of the 239 clinical specimens received in transport media in this study (ie, split-sample study), with the exception of one specimen that lacked sufficient residual volume to be performed on the Xpert Xpress SARS-CoV-2 (Cepheid) assay. In total, 168 of the specimens were confirmed to contain the SARS-CoV-2 virus by the demonstration of two or more positive tests (ie, at least two unique assays were positive on the same specimen), whereas 70 specimens were found to not contain the virus by the method described. Although input volume into the nucleic acid amplification reactions for each assay varied, which may affect sensitivity, these assays were all performed according to either our laboratory standard operating procedures or the manufacturer's guidelines that were current at the time of this study. It is noted that the FDA EUA for the ID Now COVID-19 (Abbott) has subsequently been changed to exclude specimen in transport media as an acceptable substrate for testing with the claim that the dilutional effect of placing the specimen into transport media reduces the sensitivity of the assay.[8] Although this may be true, such a dilutional effect would be the same for all assays included in this comparison. We, therefore, conclude that this is a sound assessment of the analytical sensitivity and specificity of these assays.

The comprehensive design and assessment of the CDC 2019 nCoV Real-Time RT-PCR Diagnostic Panel is available for further review, which, like our study, describes the excellent sensitivity and specificity of this assay.[2] Similarly, a thorough evaluation of the components of the TIB MOLBIOL/Roche z 480 assay is available for those interested in further reading.[9] It has been our experience that the elevated indeterminate rate experienced for the TIB MOLBIOL/Roche z 480 assay in this study does not reflect the indeterminate rate of this assay in routine practice. The only difference in this study and routine use was a single freeze-thaw cycle, which we hypothesize may have contributed to this elevated rate through target degradation.

The excellent performance of the Xpert Xpress SARS-CoV-2 (Cepheid) assay described in this study is like that described in multicenter trials by others.[10,11] A comparison between the Xpert Xpress SARS-CoV-2 (Cepheid) and the standard-of-care RT-PCR for SARS-CoV-2 demonstrated a 99.5% positive percent agreement and a 95.8% negative percent agreement, with the majority of discrepant results resolved by a third RT-PCR in favor of the Xpert Xpress SARS-CoV-2 (Cepheid).[10]

The assays that lacked a nucleic acid extraction step produced more false-negative reactions than assays that included this step in our study. Although there are a variety of reasons for different amplification efficiencies between assays, such as primer design and nearest neighbor influences, among others, the presence of amplification inhibitors is a common cause of decreased efficiency or even inhibition in assays that begin with a simple lysis step rather than purified nucleic acid extract.[12] Wilson et al[12] demonstrated this by examining the same concentration of target and the same amplification assay after processing by several different methods. An eloquent study by Fung et al[13] determined the analytical limits of detection for seven nucleic acid amplification SARS-CoV-2 assays using material quantitatively characterized using digital droplet PCR. In this study, the Roche cobas, Abbott m2000, and Hologic Panther Fusion assays had lower limits of detection than both the Simplexa COVID-19 Direct Kit (DiaSorin) assay and the ID Now COVID-19 (Abbott) assay.[13] Importantly, the variable performance of the CDC 2019 nCoV Real-Time RT-PCR Diagnostic Panel was related to the type of extraction used.[13]

The Simplexa COVID-19 Direct Kit (DiaSorin) assay has also been assessed in a number of studies.[14–16] Two of these demonstrated a greater sensitivity or positive percent agreement compared with that described here.[15,16] This is likely explained by the significant number of patients entering convalescence in our cohort, which was similar to that described by Lieberman et al.[14]

The limited comparative sensitivity of the ID Now COVID-19 (Abbott) has been described by others.[17,18] A comparison of the ID Now COVID-19 (Abbott) with the Abbott RealTime SARS-CoV-2 assay (Abbott) on the m2000 showed an overall positive percent agreement of only 75%.[18] Similarly, Basu et al[17] reported that the Abbott ID Now COVID-19 missed one-third of the specimens that tested positive by the Xpert Xpress SARS-CoV-2 assay (Cepheid) when using nasopharyngeal swabs in viral transport media, similar to our study, and a positive percent agreement of 45% when dry nasal swabs were tested directly.

To understand how these assays might perform in our diverse patient population, the specimens tested in the study were not selected from any particular population or based on any set of clinical characteristics. Rather, they were a true representation of the specimens received in the laboratory at the time of the study. Outpatients represented the majority of those tested, and as such, the presence of pneumonia, hospital admission, and ICU stay were relatively low in frequency. Different results might be obtained in a population of individuals with more severe illness.

Our study included a high proportion (38.9%) of HCPs, some of whom were tested more than 2 weeks after onset of illness when viral loads would presumably be declining. Specimens were collected from HCPs in the outpatient setting with nasal or nasopharyngeal swabs and submitted in VTM. The swab/transport medium was a variable chosen for inclusion in the multivariate analysis, as we were interested in ensuring that there were no effects of locally produced transport medium not detected in initial validation studies. However, we noted that the graphs of Ct vs days from onset to specimen collection demonstrate the correlation of decreasing viral load as number of days from first symptoms increases (Figure 1). If some HCPs were tested as symptoms were waning, it stands to reason that lower viral loads were detected and false negatives would be likely.

Use of nasal swab/VTM predominated as the NP swab/UTM collection kit was in short supply, and the latter was preferentially used for inpatients and those seen in the ED. In our study, the use of nasal swab/VTM was independently associated with false-negative results in the ID Now COVID-19 (Abbott) assay. As the swab type and transport medium were always used together, it is difficult to distinguish independent effects of each component. As discussed above, the dilutional effect of transport medium may have affected sensitivity, but other considerations that may affect compatibility must also be considered. For the Simplexa COVID-19 Direct Kit (DiaSorin) assay, although transport medium was associated with false-negative results on univariable analysis, the association was not significant on multivariable analysis when adjusted for viral load.