Test sensitivity and specificity influence cost of pandemic

Introduction

By Arundhati Rao, MD, PhD; Briget M da Graca, JD, MS; Nguyen Nguyen, PhD; Alejandro C. Arroliga, MD; William Koss, MD; Eduardo Castro, MD, MPH; Shekhar Ghamande, MD; Alita Risinger; Manohar Mutnal, PhD; and Amin. A. Mohammad, PhD.

A major concern of the COVID-19 pandemic is the financial burden imposed on the U.S. healthcare system, which has been expressed by elected officials, healthcare economists and health professionals.^1,2,3

Monte Carlo simulation analysis suggests that if 20% of the U.S. population were to be infected, there could be a median of 11.2 million hospitalizations, 2.7 million ICU admissions, 1.6 million patients requiring a ventilator, 62.3 million hospital bed days, and $163.⁴ billion in direct medical costs over the course of the pandemic.4 An analysis performed by Kaiser Family Foundation estimated the average cost of COVID-19 treatment for a patient with employer-based insurance and without complications at $9,763, and this could double or more with complications.⁵

Laboratory tests help diagnosis multiple diseases, including COVID-19, such as a positive reverse-transcriptase polymerase chain reaction (rtPCR) to confirm diagnosis. Based on symptom severity, a patient may go home to self-quarantine for 14 days or may be admitted to a COVID-19 care unit. The rtPCR test results could be true positive (TP), true negative (TN), false positive (FP) or false negative (FN). The probability of each is determined by the test sensitivity and specificity, which has a huge impact on how a patient is treated, a fact often overlooked. The most routinely used rtPCR test has a sensitivity ranging from 60–90%, depending on numerous pre-analytic and analytic variables, including when the patient is tested after symptom onset.⁶

Sensitivity defines the proportion of patients with the disease who will have a positive result, which is useful in ruling out a disease with a negative test. On the other hand, the specificity of a test is the proportion of people without the disease who will have a negative result, which is useful for ruling in a disease if a person tests positive.⁷ From a clinician’s perspective, positive and negative predictive values for a given test are the most important parameter. The positive predictive value (PPV) of a test is the proportion of people with a positive test result who actually have the disease. The negative predictive value (NPV) of a test is the proportion of people with a negative test result who do not have the disease. Both PPV and NPV are highly dependent on the prevalence of a disease in the population. A test with good sensitivity will have moderate to low PPV if it is used in locales with low disease prevalence.

Responding to the pandemic, the U.S. Food and Drug Administration (FDA) started issuing emergency use authorizations (EUAs) on February 4, 2020,⁸ resulting in a plethora of rtPCR and serological tests flooding the marketplace.⁹ Since laboratory tests play a pivotal role in triaging patient care, the PPV of the test has a significant impact on overall cost burden. With such a wide range of tests available for COVID-19, the varying sensitivities and specificities of these tests will affect the overall cost for treating patients in emergency departments suspected of having COVID-19. While many publications discuss the diagnostic impact of test characteristics on a patient’s outcome, the impact of test sensitivity and specificity on overall treatment cost for COVID-19 patients has not yet been addressed.^{10,11, 12}

The emergency department (ED) at our tertiary academic medical center (Baylor Scott & White Medical Center – Temple, TX) annually treats 102,000 patients, and approximately 40% (40,800) of these patients have symptoms, per guidelines from the Centers for Disease Control and Prevention (CDC), suspicious for COVID-19. We found the simulated impact of differing test sensitivities and specificities on hospital charges for suspected COVID-19 patients in the population.

Exhibit 1 lists the assumptions that were used to perform deterministic simulation analysis and calculate predicted charge estimates.

Cost analysis of COVID-19 patients

Between the months of April 1st–June 30th, 2020, 170 patients treated in the ED had the ICD-10 code for COVID-19 (U07.1). Of the 170 patients, 153 were discharged from an ED within 24 hours after a negative test result, and the remaining 17 were admitted for observation or treatment after being confirmed positive for COVID-19 by rtPCR. Exhibit 2 compares the charges/LOS and LOS for COVID-19 patients treated in the ED and discharged with those admitted as inpatients. The charges for the 153 patients discharged from the ED ranged from $364 - $10,130, with a median of $3,208. Expectedly, charges for patients admitted to COVID-19 wards were significantly higher, with a median of $7,815 per day of hospitalization, ranging from $4,596 - $8,446. Median LOS for a COVID-19 patient was found to be 3.4 days with a minimum and maximum of 2 and 10 days. Based on these median LOS and charges, estimated charges for a TP, FP, TN and FN patient were $26,571 (3.4 days x $7,815.0), $ 15,630 (2 days x $7,815.0), $3,208 and $29,779 (charge for TP + charge for TN).

Exhibit 3 graphs that with an ideal test at a sensitivity and specificity of 100%, the total financial burden of hospital charges would be $132.8 million dollars per annum if disease prevalence was maintained at 0.2%. However, if the prevalence increased to 10%, there would be a 68% increase in charges to $226.9 million dollars.

False negative and false positive impacts

For a diagnostic test to accurately identify a patient as having, or not having, a disease is clinically important. A false positive result can cause anxiety and result in patients undergoing treatment for a condition they do not have, incurring all the risks and expenses involved. Alternatively, a false negative can result in timely intervention being missed, worsening disease, requiring more resource intense intervention, disability or even death. In the pandemic, test results drive not only patient treatment decisions but also isolation and quarantine requirements of patients and their contacts. False negatives carry the additional burden of individuals remaining in the community and infecting others who will then require treatment.

Results demonstrate that, even at a single healthcare system, the hospital charges for a declining positive predictive value are substantial —whether driven by charges associated with treating false positives (such as declining specificity in high or low disease prevalence), or by the costs of later patient admissions with false negative results, plus the people they infected (as in the case of declining sensitivity in high disease prevalence). To maximize patient benefit and avoid unnecessary spending, test characteristics, as well as disease prevalence, need to be considered when choosing an appropriate test and/or when interpreting results and deciding whether a patient should be considered as having COVID-19 for contact tracing and disease containment purposes.

Current rtPCR test sensitivity ranges from 60% to 90% and specificity of 99.0 – 99.7%. Improving test sensitivity from 60% to 99% would result in savings of $0.96 and $47.38 million dollars for low- and high-prevalence scenarios in one year at this single tertiary-care medical center. However, improving test specificity from 60% - 99.7% would result in bigger savings of $202.34 and $183.67 million dollars for low- and high-prevalence instances. A test with low specificity will result in higher numbers of false positive results for patients, who would be hospitalized for 1-2 days, before being confirmed as COVID-19 negative and discharged.

Previous research examined costs associated with false positives in mammograms;¹³ prostate, lung, and ovarian cancer screenings;¹⁴ and radiographic interpretations in the pediatric emergency department.¹⁵ Costs of false negatives have been estimated, for example, for human epidermal growth factor receptor 2 (HER2) testing in patients with breast cancer.¹⁶ These studies report increased costs associated with inaccurate results, but differ considerably in terms of context from the examination of COVID-19 testing. Most importantly, they examined the diagnosis of conditions with relatively stable prevalence, creating a stable positive predictive value for a diagnostic procedure with a given sensitivity and specificity and in conditions not involving contagious pathogen; meaning, there is no risk of people with false negative results then unknowingly infecting others.

In contrast, COVID-19 is highly contagious, with an unstable prevalence, differing geographically and over time, creating challenges as localized “hot spots” develop and are controlled through various non-pharmaceutical interventions. A test with a particular specificity and sensitivity may provide adequate diagnostic accuracy to successfully identify and control an outbreak in one community without incurring excessive unnecessary costs. However, in another community with a different disease prevalence, it is woefully inadequate and results in unnecessary treatment costs associated with false-positive patients — or releases so many false negatives into the population that “test and trace” containment fails.

Conclusion

Test selection in the United States is based largely on the availability of tests and supplies needed to run them.¹⁷ This is likely to continue when there are shortages or disruptions in the supply chain. Results demonstrate that failing to take local disease prevalence into account when choosing a test or interpreting results can incur substantial, unnecessary charges.

Analysis shows that when disease prevalence is low (≤ 0.2%), it is reasonable to have a test with high specificity (≥ 99.5 %), while allowing some flexibility in sensitivity (ranging from 95.0% - 60.0%). However, when disease prevalence increases to ≥ 10%, the best option is to have tests with both sensitivity and specificity as close to 100% as possible. Healthcare providers and public health officials should consider strategies to mitigate the risks of inaccurate results, such as repeat testing and giving greater weight to symptoms and epidemiologic risk factors.

The authors are executives, directors, managers, physicians, and laboratorians at Baylor Scott & White Medical Center – Temple, TX.

References:

Carmody S. Gov. Whitmer asks for federal funding for National Guard COVID-19 operations. Michigan Radio. https://www.michiganradio.org/post/gov-whitmer-asks-federal-funding-national-guard-covid-19-operations. Published March 19, 2020. Accessed June 17, 2021.
Scott D. Coronavirus is exposing all of the weaknesses in the US health system. Vox. https://www.vox.com/policy-and-politics/2020/3/16/21173766/coronavirus-covid-19-us-cases-health-care-system. Published March 16, 2020. Accessed June 17, 2021.
Cohn J. The next big coronavirus worry: Can hospitals handle the influx? HuffPost. https://www.huffpost.com/entry/coronavirus-outbreakhospital-icu-masks-shortages_n_5e6521f9c5b6670e72f9b902. Published March 10, 2020. Accessed June 17, 2021.
Bartsch SM, Ferguson MC, McKinnell JA, O’Shea KJ, Wedlock PT, Siegmund SS, et al. The potential health care costs and resource use associated with COVID-19 in The United States. Health Aff (Millwood). 2020;39(6):927-35. doi:10.1377/hlthaff.2020.00426.
Rae M. Potential costs of COVID-19 treatment for people with employer coverage. Peterson-KFF Health System Tracker. https://www.healthsystemtracker.org/brief/potential-costs-of-coronavirus-treatment-for-people-with-employer-coverage/. Published April 14, 2020. Accessed June 17, 2021.
To KK, Tsang OT, Leung WS, Tam AR, Wu TC, Lung DC, et al. Temporal profiles of viral load in posterior oropharyngeal saliva samples and serum antibody responses during infection by SARS-CoV-2: an observational cohort study. Lancet Infect Dis. 2020;20(5):565-74. doi:10.1016/S1473-3099(20)30196-1.
Akobeng AK. Understanding diagnostic tests 1: sensitivity, specificity and predictive values. Acta Paediatr. 2007;96(3):338-41. doi:10.1111/j.1651-2227.2006.00180.x.
Center for Devices and Radiological Health. Coronavirus Disease 2019 (COVID-19) EUA. U.S. Food and Drug Administration. https://www.fda.gov/medical-devices/emergency-use-authorizations-medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices. Accessed June 17, 2021.
Center for Devices and Radiological Health. In Vitro Diagnostics EUAs. U.S. Food and Drug Administration. https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas. Accessed June 17, 2021.
Quan-Xin Long, Bai-Zhong Liu, Hai-Jun Deng, Gui-Cheng Wu. Antibody responses to SARS-Cov-2 in patients with COVID-19. Nature Medicine. Available from https://doi.org/10.1038/s41591-020-0897-1.
Mei San Tang, Karl G. Hock, Nicole M. Longsdon, Jennifer E. Hayes, Ann M. Gronowski, Neil W. Anderson, Christopher W. Farnsworth. Clinical Performance of Two SARS-CoV-2 Serologic Assays. Clin Chem 2020;66(8):1055-62. doi:10.1093/clinchem/hvaa120.
David Jacofsky, Emilia M Jacofsky and Marc Jacofsky. Understanding Antibody Testing for COVID-19. The Journal of Arthroplasty. Available from: https://doi.org/10.1016/j.arth.2020.04.055. doi:10.1016/j.arth.2020.04.055.
Ong MS, Mandl KD. National expenditure for false-positive mammograms and breast cancer overdiagnoses estimated at $4 billion a year. Health Aff (Millwood). 2015;34(4):576-83. doi:10.1377/hlthaff.2014.1087.
Lafata JE, Simpkins J, Lamerato L, Poisson L, Divine G, Johnson CC. The economic impact of false-positive cancer screens. Cancer Epidemiol Biomarkers Prev. 2004;13(12):2126-32.
Walsh-Kelly CM, Hennes HM, Melzer-Lange MD. False-positive preliminary radiograph interpretations in a pediatric emergency department: clinical and economic impact. Am J Emerg Med. 1997;15(4):354-6. doi:10.1016/s0735-6757(97)90123-6.
Garrison LP, Jr., Babigumira JB, Masaquel A, Wang BC, Lalla D, Brammer M. The Lifetime Economic Burden of Inaccurate HER2 Testing: Estimating the Costs of False-Positive and False-Negative HER2 Test Results in US Patients with Early-Stage Breast Cancer. Value Health. 2015;18(4):541-6. doi:10.1016/j.jval.2015.01.012.
Behnam M, Dey A, Gambell T, Talwar V. COVID-19: Overcoming supply shortages for diagnostic testing. McKinsey & Company. https://www.mckinsey.com/industries/pharmaceuticals-and-medical-products/our-insights/covid-19-overcoming-supply-shortages-for-diagnostic-testing#. Published July 15, 2020. Accessed June 17, 2021.
Who’s going to pay for Covid-19 treatment? Advisory Board. https://www.advisory.com/research/health-plan-advisory-council/members/expert-insights/2020/whos-going-to-pay-for-covid-19-treatment. Published April 20, 2020. Accessed June 17, 2021.