QC practices that prevent inaccurate results from reagent lot variances

Aprimary goal in clinical laboratories is to produce accurate test results. In fact, after labs arrive at the sweet spot of desirable exactitude, they struggle to keep conditions as stable as possible. Much of the everyday job in the lab consists of managing change, expected or not. Among the most challenging changes are those affecting materials: samples, calibrators, internal and external controls, and the reagents used to perform the tests themselves.

Many of these changes can be detected and evaluated by the lab’s analytical quality control (QC) program but detecting differences in test results caused by changes of test reagent lots constitutes a greater challenge because internal lab QC frequently fails to appropriately reflect the changes in patient sample results obtained using different manufacturer lots of the reagents used.

Reagent lot variances explained

Clinical laboratory reagents contain a variety of substances incorporated for a variety of reasons. Some are related to the intrinsic sensitivity and specificity of the measurement process. For example, in immunoassays, reagent polyclonal antibodies are generally extracted from animal blood, while monoclonal antibodies are purified from tissue culture media. Other components affect solubility, storage stability, viscosity and other properties needed to test samples using the automated equipment of modern clinical laboratories. When current batches of reagents become unavailable because of inventory exhaustion or other reasons, new batches are manufactured. Due to the complexity of these reagents, the resulting mix can vary in reactivity from those of other lots because of changes in the components. They are manufactured internally or purchased from secondary suppliers who might have inadvertently modified the composition of their product, possibly because of unwitting changes made by their own tertiary providers. Because of the minuscule concentrations and challenging specificity of the substances being measured, even a slight difference in one or more of these multiple components can change the relationship between the measurand concentration and the measurement signal generated by the instrument, and this change can be different when using different samples, such as QC and actual patient samples.

Back in 2011, Miller et al.¹ reported that about 7% of the net differences in QC material results seen after 1,483 reagent lot changes affecting 82 different tests and 7 instruments were significantly larger than the changes observed in patient samples because of limitations in the commutability of QC materials. These differences biased the QC results of various tests about equally in both directions. Similar observations have been reported since then and shown to negatively impact the care of patients, especially those being monitored over time frames that exceed reagent lot longevity, such as hemoglobin A1c and prostate-specific antigen.²

Assessing the effect of reagent lot changes

Commutability is a property of reference materials, including quality control samples, such that the differences between results when tested using two or more different test procedures for the same analyte are close to the differences seen on patient samples.³ Thus, the relative difference between results of patient samples tested with two separate lots of reagents would be comparable to the differences in results obtained using samples of commutable materials. But commutability of QC materials is not a given because the composition of QC materials differs significantly from that of patient samples: while analyte content in patient samples is highly variable and their stability is ephemeral, QC samples are manufactured with the purpose of maintaining a fixed and stable level of analyte.

For this reason, assessment of the effect of reagent lot changes on test results needs to include testing of patient samples. This recommendation is emphasized in the 2nd edition of the Clinical Laboratory Standardization Institute (CLSI) EP26 Guideline, which was published recently,⁴ and in a February 2022 publication of the European Federation of Clinical Chemistry and Laboratory Medicine (EFCLM).⁵

Both guidelines compare the magnitude of the effect of reagent lot changes against absolute limits of acceptable bias. EP26 refers to these limits as the “critical difference.” The maximum acceptable differences between the results obtained using two different reagent lots is derived from the total analytical error allowable (TEa).^6,7 In the EFCLM guideline, the total measurement uncertainty (MU) that is acceptable is split into within-lot and among-lot sources, and the maximum acceptable size of the among-lot MU component is derived from the within-subject biological variation. Both methods normalize the differences in patient sample results against the within-lot imprecision estimated by the lab QC, but while EP26 considers the two most recent adjacent lots, the EFCLM guideline estimates the within-lot and the among-lot MU components along the full QC history of the test.

The practicality of any approach to the evaluation of reagent lot changes should consider the working environment of a busy clinical lab, which is looking more and more like the 1950’s Lucy and Ethel chocolate factory conveyor belt. Relatively few reagent lot changes will have significant impacts on patient test results, and only few tests experience lot-to-lot differences that can affect multiple reagent lot changes, but these changes can affect test results enough to lead to negative patient outcomes. For that reason, and because it could affect contrived samples, such QC and proficiency testing materials can exaggerate the lot-to-lot changes and may miss changes. The laboratory needs to rule out lot switch–related biases using a practical protocol that is based on patient sample testing and can be executed timely.

Aware of this challenge, EP26 breaks down the effort in stages: preparatory, execution, calculation. The preparatory stage includes defining applicable TEa limits (which can be the same as those used for QC result interpretation) ^6,7 and selecting the patient samples to be used. More than 10 of its 122 pages consist of tables that list the number of patient samples to use according to the desired statistical power (tolerance for varying rates of false lot rejection or failure to detect significant differences), and a similar number of pages contain illustrative examples. In many lot-to-lot reagent validation studies, a handful of samples would suffice, as it has been reported by others.8 Thus, if the preparatory work is completed in advance of the actual reagent lot switch, its execution can be performed expeditiously without much impact on testing activities.

Peer laboratories can enhance their assessments by the aggregation of their results. A recent article in Clinical Chemistry and Laboratory Medicine describes the one-year experience of 27 laboratories participating in a Norwegian Organization of Laboratory Examinations program that gathered patient sample results from the evaluation of 28–29 reagent lot changes for 5 tests. The mean differences between lots ranged from 2.2% for HbA1c to 5.5% for D-dimer. The article proposed that information about multiple lot changes from different clinical laboratories could be accumulated and shared to allow simplification of lot evaluations in individual laboratories and provides real-world data about lot-to-lot variations.⁹

Achieving reliability

 Like with so many lab issues, the devil can be in the details. For that reason, the laboratory’s protocol should be able to rule out other causes of bias that coincide with reagent lot changes, such as shifts that could have taken place between the last time QC material was tested, but prior to reagent lot change. These can be ruled out by retesting QC material immediately prior to the lot change. Another trivial cause of observed lot-to-lot differences could be attributable to the lot calibrations, such as small calibration biases affecting the two reagent lots in opposite directions adding up to larger lot-to-lot net differences. In the case of immunoassays that use nonlinear calibrations, these biases might affect various analyte concentrations differently. Temporary increases in imprecision can also become fixed in the form of longer-term biases when calibrations are performed during those periods.

The role of reagent manufacturers in ensuring accurate test results through proper calibrator value assignments traceable to standard or reference materials cannot be underestimated.¹⁰ Their interest in the accuracy of test results coincides with that of the laboratories, if only because of its implied liability risk.¹¹ However, manufacturer lot release testing might not reflect all of the related variables, such as the duration and conditions of reagent shipment, use of less common instrument models, differences in reportable ranges limits, and the variability of patient populations.

Because of these limitations in manufacturer lot release testing and because contrived samples such as those used for internal and external QC are not necessarily commutable and can exaggerate or hide lot-to-lot changes, verification of the comparability of results across different reagent lots using patient sample results constitutes an essential tool for the long-term maintenance of the exactitude of clinical laboratory testing.

References

1. Miller WG, Erek A, Cunningham TD, Oladipo O, Scott MG, Johnson RE. Commutability limitations influence quality control results with different reagent lots. Clin Chem. 2011;57(1):76-83. doi:10.1373/clinchem.2010.148106.
2. Loh TP, Sandberg S, Horvath AR. Lot-to-lot reagent verification: challenges and possible solutions. Clin Chem Lab Med. 2022;60(5):675-680. doi:10.1515/cclm-2022-0092.
3. Miller WG, Myers GL. Commutability still matters. Clin Chem. 2013;59(9):1291-1293. doi:10.1373/clinchem.2013.208785.
4. CLSI. User Evaluation of Acceptability of a Reagent Lot Change. CLSI guideline EP26. Clinical and Laboratory Standards Institute. 2022.
5. van Schrojenstein Lantman M, Çubukçu HC, Boursier G, et al. An approach for determining allowable between reagent lot variation. Clin Chem Lab Med. 2022;60(5):681-688. doi:10.1515/cclm-2022-0083.
6. Ricós C, Fernandez-Calle P, Perich C, and Westgard JO. Internal quality control–past, present and future trends. Advances in Laboratory Medicine/Avances en Medicina de Laboratorio. 2022;0(0). doi:10.1515/almed-2022-0029.
7. Westgard JO, Westgard SA. Quality control review: implementing a scientifically based quality control system. Ann Clin Biochem. 2016;53(1):32-50. doi:10.1177/0004563215597248.
8. Kim S, Chang J, Kim SK, Park S, Huh J, Jeong TD. Sample size and rejection limits for detecting reagent lot variability: analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data. Clin Chem Lab Med. 2021;59(1):127-138. doi:10.1515/cclm-2020-0454.
9. Solsvik AE, Kristoffersen AH, Sandberg S, et al. A national surveillance program for evaluating new reagent lots in medical laboratories. Clin Chem Lab Med. 2022;60(3):351-360. doi:10.1515/cclm-2021-1262.
10. Theodorsson E. The six traceability models of ISO 17511:2020. Jctlm.org. Last updated March 23, 2022. Accessed June 2, 2022. https://www.jctlm.org/media/1188/2022-03-23-the-six-traceability-models-of-iso-17511_2020.pdf.
11. Quest diagnostics to pay U.S. $302 million to resolve allegations that a subsidiary sold misbranded test kits. Justice.gov. Published April 15, 2009. Accessed June 2, 2022. https://www.justice.gov/opa/pr/quest-diagnostics-pay-us-302-million-resolve-allegations-subsidiary-sold-misbranded-test-kits.

Samuel B. Reichberg, MD, PhD completed his medical residency in laboratory medicine, his PhD in Molecular Biophysics and Biochemistry, and postdoctoral fellowships at Yale University. He is currently director of the Pandemic Response Laboratory in New York City and is a fellow, educator, and inspector for the College of American Pathologists.