"Clinical pathologist Ramy Arnaout, MD, DPhil, Associate Director of the Clinical Microbiology Laboratories at BIDMC, discusses how the immunome — all of the genes collectively expressed by an individual's immune cells — holds the potential to provide researchers and physicians with unprecedented insight into an individual's health. Collecting that information from large numbers of patients could one day facilitate diagnostics via a near-universal blood test and pave the way to targeted therapies for a wide variety of conditions.
https://www.bidmc.org/about-bidmc/news/2021/05/immunomics-conversation-future-diagnostics-ramy-arnaout"

Online today in Journal of Clinical Microbiology.  
https://jcm.asm.org/content/early/2019/09/13/JCM.01270-19
DOI: 10.1128/JCM.01270-19
Copyright© 2019 American Society forMicrobiology.
Link to final accepted manuscript.

Abstract

There are now several new antibiotics available to treat multidrug-resistant pathogens, and susceptibility testing methods for these drugs are increasingly available at the time of drug approval. However, lack of clarity around verification requirements remains a formidable barrier for introducing such testing in clinical laboratories, making these drugs practically unavailable for patient use.  We propose a change in framework for bringing in testing for new antibiotics focusing on quality control rather than underpowered verification studies.
 
Text

Emergence of multidrug-resistant pathogens increasingly limits treatment options for patients. However, there have been several encouraging developments in addressing this issue.  Specifically, there are an increasing number of incentives for antimicrobial development, with new ones under discussion (1). Encouragingly, several new antibiotics have come to market in the past few years. However, to a major extent, new antimicrobials will not be used clinically in the absence of antimicrobial susceptibility testing (AST) methods in place to confirm activity.  Although in the past there was up to a several year delay in availability of FDA-cleared AST methods after drug launch, recent coordinated review by the US Food and Drug Administration’s Center for Devices and Radiological Health and Center for Drug Evaluation and Research has led to the availability of cleared AST methods at the time of drug approval (2). 

However, there remains a major barrier to the practical availability of this testing in clinical laboratories. Specifically, there is lack of clarity regarding verification requirements for bringing in testing for each new antibiotic using platforms and methods that are already established in the clinical laboratory. Without clear guidance, an assumption in the field is that a new accuracy and precision verification study must be carried out for each new drug. This places an undue burden on clinical labs and has been a hindrance to offering testing for new antibiotics. Without availability of testing in local laboratories, the antibiotics are not adopted in hospital systems and individual clinician practices, to the detriment of patients.  

​For instance, even though disk diffusion testing may already have been in use for years in a clinical laboratory, it is a common interpretation that a new verification study will have to be performed prior to bringing in a disk method for a newly approved antibiotic. This will require obtaining susceptible and resistant isolates, performing verification testing, collating data, and completing a verification write up. Standards in the field suggest testing, for example, 30 such isolates (3). In our experience, this study may require up to two days of technologist and laboratory director time. Furthermore, only recently have such isolates with defined resistance patterns for new antibiotics become readily available through efforts such as the FDA and CDC Antibiotic Resistance Isolate Bank (4). A pharmaceutical company may be able to facilitate access to such isolates, but the clinical lab must still proactively investigate isolate availability and address paperwork, shipping, and storage. Although isolates may be freely available, there is still significant effort and delays involved in obtaining them. 

With this burden and lack of regulatory clarity, the reality is that most labs will not bring in testing for new antibiotics. A verification study for each new drug is far beyond the capacity of smaller labs. The alternative, sending isolates to a reference laboratory for susceptibility testing, often does not provide actionable results for a week, which is not a desirable situation for patients and their care providers (5). In some cases, reference laboratories do not even offer testing for recently introduced antimicrobials. The result is that new antibiotics are not being used and clinicians are forced to fall back on drugs that, although potentially active (for example, colistin), may not have optimal activity or side effect profiles.

​The requirement for laboratories to perform an accuracy and precision study using the common rule of thumb of 30 isolates is extremely underpowered from a statistical perspective (6). For example, the FDA's guidance on approval of AST devices highlights that such a small study would be inappropriate to characterize the very major, major, and minor error rates for a method (7).  

Clearly, therefore, the purpose of a verification study is not to replicate the studies required for FDA submission. Then what is the purpose? We should define this purpose clearly, rather than simply employing terms such as accuracy and precision, as in the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88), without considering their relevance to the real goal of readily available, clinically useful AST results. Our interpretation is that a verification study should be used to show that (1) a laboratory can adequately perform a technique i.e., that operator-dependent variables do not compromise integrity of testing results, and (2) operator independent characteristics of the method are not compromised by placement of the method in a new laboratory environment. The former is most relevant when evaluating techniques such as disk diffusion and gradient methods, while the latter is particularly important when evaluating automated systems such as the Vitek 2 where subtle perturbations to, for example, instrument mechanics and optics at least theoretically may create systematic bias in results.  

Both operator-dependent and independent reliability can be established when the method is first brought into the lab using a subset of antibiotics. An abridged accuracy and precision study at this time and in this context serves as a check to ensure that the method generally performs according to specification. For operator-dependent methods, it ensures that technologists are adequately trained to consistently perform the method. The verification study does not recapitulate and cannot replace the in-depth, statistically powered study performed by the manufacturer along with stringent expert review required for clearance of the AST method. We are not proposing changes to current standards for verifying new methodology when first brought into the clinical lab.

However, with the goal of a laboratory verification study clearly defined, it is our opinion that bringing in testing for each new antibiotic, using a method previously established in the clinical laboratory, should not require an additional verification study. In the case of disk diffusion and gradient diffusion methods, the ability of the laboratory to adequately perform the technique has previously been established. In the case of automated systems, the operator dependent and independent reliability have been previously confirmed. As such, quality control as recommended in the antimicrobial package insert and/or by CLSI should be sufficient to ensure adequate AST performance without need for any additional pre-implementation studies. 

This common sense approach will allow immediate adoption of testing of new drugs and benefit patients and pharmaceutical companies alike.  Importantly, however, clinical laboratories need to have confidence that Center for Medicare and Medicaid Services (CMS) and deemed accreditation organizations such as the College of American Pathologists will consider the absence of the additional and nonsensical verification studies for new drugs on already existing platforms in line with the letter and spirit of CLIA' 88 requirements.  Official clarification in this area would be immensely appreciated. We hope that this article may be a reference for clinical laboratories to justify this approach to laboratory inspectors in the interim. We make further note that we (6) and at least one other set of authors, more tentatively (8), have previously suggested such an approach.  

The ecosystem for new antimicrobial development is, to put it mildly, fragile. Several pharmaceutical companies have withdrawn from the antimicrobial development space or declared bankruptcy in the past year (1). Antibiotics are at the forefront of personalized medicine.  Medications for diabetes and high blood pressure, for example, don't require a test up front to determine whether they will work for a specific patient, but antibiotics do. Removing barriers for offering susceptibility testing for new antibiotics will therefore serve two purposes: providing timely access to potentially life saving therapy and supporting pharmaceutical investment in a critical area of personalized medicine that has an unpredictable return on investment.

In conclusion, to summarize our recommendations for the field:

No additional verification should be required if AST is performed using a method previously established in a clinical laboratory. The laboratory should immediately implement AST for new antimicrobials while performing recommended quality control testing. 

We believe that these recommendations will address our need to provide immediate access to new antibiotics for our patients. They will also provide pharmaceutical companies with greater confidence that antimicrobials will see immediate use after FDA approval with availability of susceptibility testing at sites of patient care and thereby encourage much needed investment in antimicrobial development. Finally, the new clarified approach will de-emphasize underpowered verification studies and refocus our efforts on quality control to ensure ongoing optimal performance of established AST methods. 
 
References
​
1. Talbot GH, Jezek A, Murray BE, Jones RN, Ebright RH, Nau GJ, Rodvold KA, Newland JG, Boucher HW. 2019. The Infectious Diseases Society of America's 10 x '20 Initiative (10 New Systemic Antibacterial Agents US Food and Drug Administration Approved by 2020): Is 20 x '20 a Possibility? Clin Infect Dis 69:1-11.
2. Humphries RM, Ferraro MJ, Hindler JA. 2018. Impact of 21st Century Cures Act on Breakpoints and Commercial Antimicrobial Susceptibility Test Systems: Progress and Pitfalls. J Clin Microbiol 56:pii: e00139-18.
3. Clark RB, Lewisnski ML, Loeffelholtz MJ, Tibbets RJ. 2009. Verification and Validation of Procedures in the Clinical Microbiology Laboratory. In Sharp SE (ed), Cumitech, vol 31A. American Society of Microbiology, Washington, D.C.
4. Lutgring JD, Machado MJ, Benahmed FH, Conville P, Shawar RM, Patel J, Brown AC. 2018. FDA-CDC Antimicrobial Resistance Isolate Bank: a Publicly Available Resource To Support Research, Development, and Regulatory Requirements. J Clin Microbiol 56:pii: e01415-17.
5. Smith KP, Kirby JE. Rapid Susceptibility Testing Methods. Clinics in Laboratory Medicine doi:10.1016/j.cll.2019.04.001.
6. Lee RA, Kirby JE. We Cannot Do It Alone: The Intersection of Public Health, Public Policy, and Clinical Microbiology. Clinics in Laboratory Medicine doi:10.1016/j.cll.2019.05.008.
7. U.S. Food and Drug Administration. 2009.  Guidance for Industry and FDA: Class II Special Controls Guidance Document: Antimicrobial Susceptibility Test (AST) Systems. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/class-ii-special-controls-guidance-document-antimicrobial-susceptibility-test-ast-systems. Accessed July 27, 2019.
8. Patel JB, Sharp S, Novak-Weekley S. 2013. Verification of Antimicrobial Susceptibility Testing Methods: a Practical Approach. Clinical Microbiology Newsletter 35:103-109.