Patient safety monitoring in international laboratories

International travel, teaching and improving laboratory quality worldwide may not describe the responsibilities of most laboratory professionals, however, medical technologists working for the Johns Hopkins University (JHU) Patient Safety Monitoring in International Laboratories (pSMILE) program have taken their clinical laboratory training and skills beyond the bench.

JHU-pSMILE is a National Institutes of Health (NIH) resource designed to evaluate and develop the capability of laboratories to participate in prevention, vaccination and therapeutic clinical studies conducted in developing countries and supported by the National Institute of Allergy and Infectious Diseases (NIAID). Designed to provide long term support to developing countries with the design and implementation of HIV/AIDS prevention and treatment research studies relevant to their populations, the program ensures the integrity and reliability of laboratory tests for monitoring the safety and efficacy of experimental products investigated in studies funded by the Division of AIDS (DAIDS) at NIAID. The pSMILE program has been operating at the Johns Hopkins University School of Medicine, Department of Pathology since the inaugural contract was awarded in 2004.

The four core functions of pSMILE are:

• Monitoring laboratories’ compliance with Good Clinical Laboratory Practice Standards (GCLP)
• Monitoring the ability of laboratories to reliably perform protocol-specified laboratory testing
• Providing laboratories with various means of assistance, guidance and training to address and prevent recurrence of deficiencies in GCLP and/or Proficiency Testing (PT) to improve quality of laboratory operations
• Hosting and maintaining a computerized data-management system and document library that includes laboratory performance data and guidance and resource documents

The JHU-pSMILE team has developed processes, standard operating procedures (SOPs) and software systems to accomplish these four core functions. Over the past sixteen years, the program has grown into an organization that is internationally recognized for its quality assurance methods. In July of 2020, JHU-pSMILE was awarded International Organization for Standardization (ISO) 9001:2015 certification. The ISO 9001:2015 standard ensures that products and services meet the needs of clients through an effective quality management system. As part of the ISO 9001:2015 certification process, JHU-pSMILE developed and implemented a quality management system to improve overall performance and maintain a high-level of quality and strong customer service.

Throughout the 16-year history of the program, the team has supported 285 laboratories in 31 different countries (Figure 1), providing expert laboratory assistance for patient-safety testing. Currently, pSMILE actively supports 144 international laboratories in 18 countries including: 30 in South America, 38 in East Africa, 41 in South Africa, seven in Southern Africa, two in the Caribbean, 25 in Asia and one in Europe.

Laboratories are selected for pSMILE support because they are performing NIAID-sponsored research for HIV/AIDS and its related co-infections and co-morbidities. Research protocols are administered through HIV Clinical Trials Networks such as the AIDS Clinical Trials Group (ACTG), International Maternal Pediatric Adolescent AIDS Clinical Trials Network (IMPAACT), HIV Prevention Trials Network (HPTN), HIV Vaccine Trials Network (HVTN), and Microbicide Trials Network (MTN).

Throughout the tenure of the program, pSMILE has been able to support the laboratories and the study participants in the network protocols of hundreds of different clinical trials. Each network focuses on a specific aspect of HIV research, as the names of the networks indicate:

• HPTN is dedicated to the prevention of HIV and has been responsible for advancing prevention protocols, such as the HPTN 052 study, which proved that early antiretroviral therapy (ART) can prevent HIV transmission. ART was named a 2011 Breakthrough of the Year by the journal Science.
• The IMPAACT network has been credited with developing protocols that greatly reduce the transmission of HIV from mother to baby through its many maternal and infant studies.
• The MTN network’s agenda focuses on microbicides and was responsible for the CAPRISA 004 study, which was the first study to show that the use of microbicides as pre-exposure prophylaxis (PrEP) could be effective.
• The ACTG network’s agenda primarily revolves around the treatment of HIV and its co-infections, especially TB. The ACTG network is credited with developing protocols proving that combinations of anti-HIV medications control HIV better than single drug regimens.
•  The HVTN network is dedicated to the search for a vaccine against HIV and has recently launched protocols that include mosaic vaccines, which is a new concept for HIV vaccines. The experimental vaccine contains a medley or mosaic of genes from various HIV subtypes. The HVTN is also working in collaboration with the HPTN on the novel concept of using Antibody Mediated Prevention (AMP) to prevent HIV infection.

pSMILE also supports non-network studies funded by DAIDS. For example, pSMILE helped laboratories that participated in the iPrEx studies, led by Robert Grant, MD, MPH at the University of California – San Francisco, who was named by Time magazine as one of 2012’s most influential people. Grant led the groundbreaking, global study that showed how existing HIV/AIDS medications could effectively be used to prevent transmission of HIV in those likely to be exposed to the virus. pSMILE also supported the laboratories participating in the Comprehensive International Program of Research on AIDS (CIPRA). The goals of the CIPRA study focused primarily on capacity building by supporting the training and infrastructure necessary for developing and sustaining ongoing research efforts.

pSMILE overview

The pSMILE resource is managed by the NIH Division of AIDS by a contracting officer’s representative (COR) who administers and directs all activities. At JHU, pSMILE activities are led by a principal investigator and a project manager who oversee daily operations and staff including:

• Ten registered medical laboratory scientists who serve as international QA/QC coordinators
• A program officer managing financial accounting and invoicing
• A programmer analyst providing website development and management
• An administrative coordinator performing administrative and clerical functions
• Information technology consultants providing targeted specialties and expertise as needed

This dedicated team of professionals comes from diverse cultural backgrounds and speak multiple languages, providing a unique basis of understanding and pertinent global sensitivity, which is beneficial to pSMILE’s international mission. Members of the current JHU-pSMILE team have worked in the field of pathology and laboratory medicine, ranging from 12 to 40 years in both the United States and globally, and have a combined total of greater than 200 years of laboratory experience. JHU-pSMILE coordinators are credentialed by a range of clinical laboratory certifying bodies, including the American Society for Clinical Pathology (ASCP) and American Medical Technologists (AMT). JHU-pSMILE also has a team member who holds certification from the American Society for Quality (ASQ) as a certified quality process analyst (CQPA). The team members’ laboratory experience is as varied as their educational and cultural backgrounds, ranging from large university hospital laboratories, commercial laboratories, international research and clinical laboratories, doctor’s office laboratories and more. They also have a wide range of knowledge and practical experience covering nearly every specialty in the clinical laboratory, including chemistry, hematology, immunology, flow cytometry, serology, microbiology, mycobacteriology, histology/cytology, and blood banking. Team members also possess advanced degrees, such as in business, distance education and biotechnology.

JHU-pSMILE coordinator training and responsibilities

To develop the skills required to be a JHU-pSMILE coordinator, new employees complete a rigorous training program. They receive training on all pSMILE internal procedures and are mentored by assigned trainers who are experienced members of the JHU-pSMILE team. By using a virtual education platform, training is standardized, comprehensive, and inclusive of all pSMILE tasks including proficiency testing review, laboratory audit review and creation of remediation action plans, instrument validation, and international laboratory visits. Since this job is unlike many in the clinical laboratory profession, it typically takes about a year to complete the training of a new pSMILE coordinator.

The day-to-day work of a JHU-pSMILE coordinator typically involves a few key tasks that almost always provide an exciting challenge. pSMILE’s approximately 144 global laboratories are divided among coordinators for everyday activities, including evaluating, analyzing and reviewing proficiency testing data, assisting with audit remediation, reviewing instrument validation and assisting with other laboratory quality issues. Coordinators also work with laboratory sites to track PT shipments as well as resolve shipping and results submission problems.

Laboratory proficiency testing

Each international laboratory is required to complete the same level of proficiency testing as is required for U.S. laboratories. pSMILE provides PT surveys, evaluates results, and follows up with a written review. Coordinators also work with laboratories on resolving PT failures with resolutions documented as part of an investigation report (IR).

pSMILE acts as a facilitator, purchasing and evaluating PT for hundreds of laboratories with a primary research focus on HIV diagnostics. As such, pSMILE is in a unique position to identify challenges and stay abreast of emerging technologies as they relate to HIV diagnostic assays and proficiency testing. In 2004 when pSMILE began monitoring HIV testing for NIH, the state-of-the-art assays were 2nd and 3rd generation enzyme-linked immunosorbent assay (ELISA) methods, with western blot being the gold standard confirmatory assay. pSMILE worked with PT providers, such as the College of American Pathologists (CAP), to ensure that peer groups and evaluation methods were sufficient for these methods. pSMILE also developed evaluation criteria for western blot bands, which were not evaluated by the PT provider, because this information was critical to HVTN protocols. As 4th generation and p24 antigen methods emerged in more and more research protocols, pSMILE again worked with PT providers, including international providers, to ensure adequate coverage and evaluation. We have now seen almost complete discontinuation of the use of western blot in research protocols, with the emergence of rapid confirmatory assays. This has provided new challenges in proficiency testing, which pSMILE has been able to overcome by using a combination of PT providers and by giving feedback to the providers to help them improve their coverage.

Many pSMILE laboratory sites participate in CAP’s PT programs. However, over the years, PT programs from other countries have also been utilized since they may provide better peer groups for international laboratories or may be better suited to assays being performed in international settings. A good example is tuberculosis (TB) testing. Since TB is more prevalent internationally than in the United States, comprehensive and robust PT material is not readily available domestically. JHU sources PT panels from Germany and France to ensure adequate PT coverage for TB culture, identification and drug susceptibility testing (DST). In addition, there are TB diagnostic methods that are widely used outside of the United States, so pSMILE utilizes PT panels from a South African source because it has developed panels that are specifically customized and validated for these methods. In the case of Interferon Gamma Release Assays (IGRA), JHU-pSMILE discovered that the U.S.-based PT was inadequate to cover the assay as it is utilized in network studies. More rigorous coverage was found utilizing panels produced in Sheffield, United Kingdom.

pSMILE also provides PT when products are not available commercially or are insufficient. For example, in response to a lack of commercially available PT, JHU-pSMILE developed vaginal wet mount microscopy PT utilizing digital images available through an online training program. This PT program assures the ability of laboratories to identify Trichomonas Vaginalis and vaginal clue cells, a critical component of one of the research protocols supported by pSMILE.

A laboratory audit performed by an independent DAIDS contractor is an entry point for new laboratories. Audits are generally performed annually for established laboratories and are based on DAIDS GCLP guidelines, which are very similar to CAP accreditation checklists. Audit reports are then sent to pSMILE for review and preparation of an action plan that guides the laboratory through the process of correcting each documented deficiency. This can sometimes be a lengthy process, involving many emails and web-based meetings between the pSMILE Coordinator and the site. The resolution of the action plan also provides opportunities for sites to improve their processes as well as for pSMILE coordinators to engage in formal and informal teaching and training.

Coordinator day-to-day work is interspersed with other responsibilities and challenges. Each coordinator generally participates in several internal and external committees working on a wide range of topics and projects. Internal committee charges include developing resources, designing pSMILE.org’s website content, developing and testing software, developing protocols for instrument validation, and preparing conference and other educational materials. pSMILE also has an internal quality assurance committee that focuses on regulatory compliance, accreditation (such as ISO 9001), and the monitoring of pSMILE internal processes to ensure quality. External committees include the JHU Pathology Department’s committees, such as the Diversity Committee or Annual Educational Symposium Committee. Team members also serve on research protocol working groups and participate in cross-network projects.

Tools of the Trade

JHU-pSMILE coordinators utilize a unique array of competencies. Extensive clinical laboratory experience is complemented with computer skills using software programs such as Microsoft Excel, Word, Teams, PowerPoint and SharePoint. Coordinators also use method validation tools and a web-based SOP management tool. In addition, JHU hosts and maintains multiple software systems that aid in managing all aspects of pSMILE functions and workflow. These web-based applications were developed in-house and include electronic document repositories, automated filing systems, and online data warehouses customized for optimal functionality and utility.

There are three primary software tools:

• Oversight Masterlist (OSML), a SQL database designed to organize and track laboratory-specific information. This is an internal application accessible only by pSMILE staff members. Examples of the comprehensive information stored for each laboratory include the location, contact information for leadership personnel, network affiliation and DAIDS oversight staff, and laboratory accreditation status.
• pSMILE.org website, a document repository that not only stores and posts laboratory-specific documents but also contains an extensive library of guidance and open-access resource documents. Resources include templates for SOPs and forms, checklists, Excel spreadsheets for calculations used in method validation studies, published articles, and web links.
• AutoSMILE, a tool built at JHU, is a database and web-based user interface for automating the review of laboratory proficiency testing data. This system provides proficiency testing reviews, summaries, and schedules that meet regulatory requirements governing clinical trials such as from the European Medicines Agency (EMA) and U.S. Food and Drug Administration (FDA). JHU has arranged for secure electronic transfer of proficiency testing data from each proficiency testing provider. JHU-pSMILE coordinators verify the providers’ evaluation of the data and supply evaluations for ungraded results. The system derives a score based on the overall evaluation and determines whether the laboratory needs to complete a report for unacceptable results. The system then generates an investigative review report that the coordinator can edit as needed and send via email to all stakeholders, including the laboratories, network personnel, and representatives from DAIDS. A valuable feature of AutoSMILE is the ability to track and assess laboratory PT performance over several years. The database generates an Excel spreadsheet for each laboratory, summarizing the performance of each protocol analyte over the previous three years.

The AutoSMILE software has been extensively validated to ensure that data integrity is maintained and the system complies with standards. AutoSMILE not only allows for the efficient use of coordinators’ time; the automated process also improves accuracy of transcription, standardizes the review of PT data, and provides timely and uniform reports to stakeholders.

International travel and teaching

Interfacing with the international laboratories on a daily basis is the key to the success of the pSMILE mission. Strong communication skills are important and cultural sensitivity goes a long way in establishing a connection and fostering collaboration. E-mail, telephone, and web-based conferencing are standard daily channels of communication. However, coordinators travel to selected laboratories throughout the year, averaging stays of one to three weeks. Laboratories are located primarily in developing, resource-constrained countries and personal safety while traveling is a priority. That is why coordinators typically travel in pairs. They spend most days in the laboratories, working side by side with our international counterparts to resolve problems, offer possible solutions, and share knowledge to help improve quality in the laboratory.

Each laboratory visit focuses on specific objectives. These may include instrument validation, assessment of laboratory testing capacity and methods, TB laboratory assessments, laboratory audit remediation, instrument/method troubleshooting, and other special assignments from the NIH sponsor. Training sessions for larger groups are often held while on-site, providing continuing education opportunities for both bench technologists and management staff. We also help and mentor laboratories to become accredited by agencies, such as CAP, South African National Accreditation System (SANAS), and ISO 15189. The work we accomplish during these trips is rewarding, and we are proud that laboratories that we have supported are recognized as having high standards of quality by the NIH, clinical trials networks, and other regional laboratories.

Team members also participate in regional and international meetings and conferences that provide an opportunity to collaborate with researchers in the field of HIV/AIDS. Each of the clinical trials networks described earlier in this article conduct annual or biannual meetings to share their research and provide training to personnel involved in their trials. The pSMILE team regularly participates in these network meetings. JHU-pSMILE team members have provided educational seminars and presentations on a wide variety of topics related to laboratory quality assurance. Presentations have included such topics as method validation, evaluation of QC ranges, technical assistance for novel TB methods, and HIV proficiency testing. The JHU-pSMILE team has also given many presentations on laboratory safety, audit preparation, and developing Quality Management Systems. The JHU-pSMILE team also has been able to participate in and contribute to research studies and publications.^1-3

Summary

The pSMILE program at JHU is a collaborative effort that ensures the quality of testing at international laboratories conducting clinical trials and studies. Although pSMILE was established to assist laboratories performing HIV/AIDS research, we are flexible enough to assist with emerging infectious diseases such as Zika and SARS CoV-2. pSMILE has provided an opportunity for interesting, fulfilling, and challenging alternative career paths in a non-laboratory, clinical research setting. We have been privileged to have a front row seat and play a role in breakthroughs in the prevention and treatment of HIV that changed the course of the history of the disease. The pSMILE resource, connecting the NIH Division of AIDS and the Johns Hopkins University, has enabled us to use our experience, skills, and education as laboratorians to go beyond the bench.

Acknowledgements

The authors thank Daniella Livnat, Division of AIDS, NIAID, NIH, for her leadership and support of the SMILE program and for critical review of the manuscript

This project has been funded in whole or in part with federal funds from the National Institute of Allergy and Infectious Diseases, National Institutes of Health, Department of Health and Human Services, under Contract No. 75N93020C00001.

References

1. Amukele TK, Michael K, Hanes M, Miller RE, Jackson BJ. External quality assurance performance of clinical research laboratories in sub-Saharan Africa. Am J Clin Path. 2012;138:720-3. doi: 10.1309/AJCP8PCM4JVLEEQR.
2. Mine M, Moyo S, Stevens P, et al. Immunohematological reference values in HIV-negative healthy adults in Botswana. Afr J Lab Med. 2011;1(1). Art. 5:1-7. doi: 10.4102/ajlm.v1i1.5.
3. Godfrey CC, Michelow PM, Godard M, et al. Improving diagnostic capability for HPV disease internationally within the NIH-NIAID Division of AIDS Clinical Trials Networks. Am J Clin Path. 2013;140:881-9. doi: 10.1309/AJCPIBIS19QIYHJY.