Risk Based QC. Patient centered QC. Risk Assessment. Patient-focused QC. Individualized Quality Control.
These terms have been popping up in articles and conferences and on the Internet over the past year. Now, as we approach the time when the Centers for Medicare and Medicaid Services (CMS) is expected to unveil the initial guidance for Individualized Quality Control Plans (IQCP), it’s time to brush up on risk management and understand what risk management aspects of the IQCP may mean for laboratories. It is likely IQCP will be a key focus for point-of-care testing (POCT), and therefore it will be the rare hospital laboratory that will not have to make a decision on how to implement it.
Most labs that use point-of-care (POC) devices have established their frequency based on the equivalent quality control (EQC) rules published by CMS in 2003. With implementation of IQCP, it is anticipated that labs will be required to revert their POC devices back to the default rule of two levels of QC per day or conduct a risk assessment to determine whether the appropriate QC for their current number and frequency is adequate or needs to be revised.
The change
According to CMS, Clinical Laboratory Improvement Amendment (CLIA) guidelines will provide an educational period during which EQC will be phased out, a period that is expected to be at least one survey cycle long (two years).1 Labs accredited by other organizations such as The Joint Commission (TJC), CAP, or COLA, will be allowed to follow their organization’s requirements, but as with the EQC, we can expect that accrediting organizations will also adopt the concepts of IQCP as an alternative to default QC.
Choosing a starting point
The first, most obvious question is which test should be reviewed by the risk assessment. You might want to start with a point-of-care test in which the current QC plan is based on EQC. Or you might want to start with a test that has a wealth of internal documentation and lab experience in order to better build your risk assessment skills. Other viable options are to select a test that you know is a poor performer or problematic, or a test for which you believe the QC plan is not consistent with its performance—either too much QC for a strong performer or too little for a weak performer. Regardless of where you start, start simply. The process has the potential to be overwhelming the first time around.
Standards and guidelines
There are three risk management documents that have current relevance and will prove helpful to laboratories:
1) ISO 14971, Application of risk management to medical devices, which was a reference document for the authors of the second document;
2) CLSI’s EP23, Laboratory Quality Control Based on Risk Management; and, of course,
3) the forthcoming CMS guidance on IQCP.
ISO 14971 is intended for use by manufacturers developing medical devices and while “not intended for clinical decision making,” there is valuable information in section H, “Guidance on risk management for in vitro diagnostic medical devices.”2 The guidance document provides valuable example lists of possible errors by users, hazards, and risk mitigation controls. As a manufacturer’s guidance document, the standard is helpful, but it is not adequate as a single source for developing a risk management plan for a clinical laboratory. The document is sold on ISO (International Organization for Standardization) standards websites.
Developed for clinical laboratories, EP23 is the first document to provide step-wise instructions. The document covers all the steps of preparing a risk management plan and provides sample checklists and specific examples for a hypothetical glucose instrument.3 The document was released in late 2011 and has been well-received. The criticism it has received has been primarily focused on the need for more precise instructions. James Westgard, PhD, has cited a few gaps in the current version, such as the lack of hazard detection as a part of the calculation of overall risk and the recognition of the role statistical quality control and instrument performance plays as a starting point in any risk management plan. At present, this is the only document available to labs that clearly lays out the path, and we can expect that the next version will address any shortcomings. This document is the best tool labs have to date and is sold on the CLSI website.
CMS officials have stated that they will base their upcoming IQCP guidance on principles of EP23. CLIA guidelines are necessarily written from a government regulatory perspective, intended to establish a baseline for minimally acceptable quality and intended to be applicable to all labs, from the most to the least sophisticated. Therefore, it is anticipated that the IQCP guidance may not be a highly technical guidance document, and labs may need to utilize EP23 or avail themselves of additional training to make an effective risk management plan. The IQCP guidelines will be released as part of the surveyor’s Interpretive Guidelines.
An additional valuable resource is Westgard QC’s Six Sigma Risk Analysis.4 This book contains a thorough review of the principles of risk assessment and management and the pros and cons of this approach. It contains examples for each step of the process and compares how the various documents address the same point.
The process
There’s no way around it: a risk assessment is going to take work. The good news is that unless you are bringing in a new test, you already have a great deal of the necessary information at hand, and you have the valuable experience of your staff that has run the test and investigated failures. The number of phases or steps of the process may vary depending on the source material, but for the purposes of an introduction to the process, let’s break it down into 10 steps that cover three phases.
Risk assessment phase
Step 1. Information review. It’s important to take the time to pull together all the information you have on the test. This could include instrument manuals, package inserts, technical bulletins, regulatory/accrediting organization requirements, QC and PT records, training procedures, prior failure investigation records, and any institutional clinical guidance on the test or analyte. Without all the information handy, the team may be tempted to rely on memory, and having it in one location will save time later when you are researching various details.
Step 2. Process mapping. Don’t try to do this alone or with just one or two others. Most business theorists suggest the most productive working groups/committees have between four and 10 members. Gather the staff who run the test method, use the test results, and collects the patient specimens, and start mapping out the process. If you can, include in the group the phlebotomists and nursing staff who collect the sample. If that’s not possible, work with them at a later time but avoid the temptation to assume you know their steps of the process; after all, you’re in the discovery mode, and it’s important to understand what really occurs, not just what is supposed to occur. The same holds true for the clinicians who order the test and act upon the result. Include them in the process and understand what they do when the test result comes back different from what they expected. Process mapping is where you will likely discover “tribal knowledge” for performing the test that may vary from the written procedure.
Step 3. Identification of hazards. Once the procedure is mapped out, you begin to ask “what can go wrong” during routine operations for each step of the process: what hazards are present. A hazard is a failure that could cause potential harm to the patient. Once again, this is best performed in a group setting. One idea will remind someone else of another hazard, and the collaboration will uncover areas for further exploration and research. Once the hazards are identified, they can then be categorized into one of five areas: Operator, Environment, System (Instrument), Reagents, or Specimen. These categories will make it easier to later identify types of controls necessary to reduce unwanted risk.
Step 4. Evaluate the risk for each hazard. This is the step where you decide how likely it is that a failure will occur and, once it occurs, how likely it is that it will lead to harming a patient and how severe that patient harm could be. This step is more subjective than the others and is likely to create a great deal of discussion among the team. It will be important to work with clinicians to understand clinical implications for an incorrect result. Given the subjectiveness of these determinations, it is recommended that the team learn as much as possible about performing this step before finalizing its assessment of the risk levels.
Step 5. Determine acceptability of risks. Once the risks are assigned, the next step is to look at severity and probability of harm to determine whether the risks are acceptable. This is an area where guidance from standards and guidelines differs, and ultimately it will be up to each lab to determine where to draw the line between acceptable and unacceptable. CLSI’s Risk Acceptability Matrix (Table 1) is but one example and one way for creating the chart. Ultimately, it will be up to the team to determine whether a given risk is acceptable or not. Sometimes the judgment can be a difficult one—for example, if a given failure is occasional and the severity of the harm is serious.
Risk control phase
Step 6. Risk reduction/control/mitgation. For any risk that is deemed unacceptable, the lab should identify ways to reduce the probability of harm, using prevention and detection methods, in order to bring the risk down to an acceptable level. This may be through a variety of means such as revised operator training, posted warnings, more robust QC rules, greater surveillance of the process, or even repeat testing for values exceeding a specified threshold. In cases where the test manufacturer indicates that certain aspects of the test are monitored through built-in controls, the lab may wish to conduct a study to verify the effectiveness of the control before including it as a control measure.
Step 7. Evaluate residual risk. Once controls are assigned, the team then reviews the risks again and re-determines the risk level. Any risk not prevented or detected 100% of the time is considered residual risk.
Step 8. Further risk control procedures. If any risks still remain unacceptable, the team must decide how to bring that test into an acceptable level for continued operation. In cases where it is not possible to mitigate the risk to an acceptable level, the team should work with clinicians to determine if the benefits of performing the test outweigh the residual unacceptable risk.
Risk monitoring phase
Step 9. Determine a monitoring plan. Since the goal of the risk management plan is to maintain or improve the quality of the test, it’s important to define a means to measure the quality to ensure that it meets the needs of the laboratory and the intended use of the test. One could argue that monitoring clinician complaints is an adequate means of monitoring quality, but this may only be true in cases where there is a close relationship between the lab and the clinical staff. In large institutions, it may be that the lab is made aware of only the most significant errors and some incorrect results are never reported back to the lab. It is better to have measures in place that are more likely to be logged or otherwise recorded.
Step 10. Write the quality control plan. A good plan does not need to be voluminous to be effective, but it should be complete enough that staff (and auditors) understand the factors that were considered in its creation. The lab may wish to include the process map and risk tables, but at a minimum they should be referenced and maintained separately. They will be invaluable later should the lab need to perform a failure investigation, evaluate a new test method, or look to improve the process of the current method.
________________
In contemplating the risk management process, a few of the steps may seem daunting to document, and certainly time consuming, but the process, if performed diligently, will provide insights that in the end will improve the quality of the test and improve patient safety.
Max Williams, MPA, is Division Global Marketing Manager, Quality Systems Division, for Bio-Rad Laboratories.
References
- CMS letter to State Survey Agency Directors, dated March 9, 2012. Implementing the Individualized Quality Control Plan (IQCP) for Clinical Laboratory Improvement Amendments (CLIA). http://www1b.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/SCLetter12_20-.pdf . Accessed September 24, 2012.
- International Organization for Standardization. ISO 14971:2007 Medical devices—application of risk management to medical devices. www.iso.org.
- Clinical Laboratory Standards Institute. Laboratory Quality Control Based on Risk Management; Approved Guideline. EP23-A. 2011. www.clsi.org/source/orders/free/ep23-a.pdf.
- Westgard JO. Six Sigma Risk Analysis. 2011. www.westgard.com.
