Successful AMU data collection

The National Animal Health Monitoring System, administered by the USDA, has collected AMU data through surveying beef feedlot and swine operations.^2,3 However, these data may be subject to recall bias, and because more granular and less biased data are desired, the AVMA Committee on Antimicrobials endorses the collection of AMU directly from animal health records or invoices from animal patients. Collection of farm-level data in animal agriculture has included from feedlots, dairies, and poultry operations.^4–6 Collection of AMU data from electronic health records has also been reported, for example, from veterinary referral hospitals and pet insurance data.^7–9 These studies are typically cross-sectional in nature at a specific moment in time on a small number of operations and are labor-intensive, even in entities that have electronic records, and the tools and knowledge needed to collect data on an ongoing basis are not widely available or do not exist. However, there are proprietary systems being built to automatically capture and provide access to reports and comparison data, such as the Pipestone Antibiotic Resistance Tracker.¹⁰ This tracker provides a system to swine farmers to “record, review, and respond” by capturing data from prescriptions and feed mills and allows for comparisons to other farmers. This type of system, along with the ongoing availability of grant funding for AMU data collection,¹¹ suggests that there may be untapped demand for automated data capture and benchmarking capabilities in some production systems or veterinary practices.

Fruitful collection of treatment outcomes data (ie, efficacy)

FDA-approved antimicrobials have been demonstrated to be effective on the basis of pivotal studies for label claims. However, these studies often include a relatively small number of animals or production units and may not capture the complexities of treatment outcomes or represent an accurate prediction of clinical success. In addition, there are many conditions for which there is no approved antimicrobial agent and for which veterinarians appropriately use drugs in an extralabel manner. In contrast, outcome measures possible with AMU data include morbidity, mortality, recrudescence, post-treatment diagnostic results, time to resolution, and client satisfaction. No outcome measure is currently known to be superior to another, although some may be more commonly accepted by partners than others. Paramount to analysis of treatment outcomes is the collection of thorough and accurate clinical histories before and after treatment. Development of antimicrobial resistance (AMR) may also be an outcome measure for veterinary AMU. However, AMR is a lagging indicator of AMU that may or may not be altered by changes in AMU, and interpretation requires detailed and unbiased epidemiologic and molecular genetic data at an aggregated level, such as production unit or veterinary practice area.

As stated in the newly adopted AVMA policy (Appendix), caution should be taken when evaluating treatment outcomes alongside AMU data: “Simply comparing antimicrobial use and treatment outcomes in individual animals or groups of animals cannot accurately establish cause-and-effect relationships.” We acknowledge that voluntary collection of outcomes in the varied animal health settings is ambitious, given that veterinarians rarely have financial or other incentives, regulatory obligations, or even sometimes follow-up information from caretakers to document animal outcomes after any therapeutic intervention, not just after antimicrobial therapy. However, to maximize the benefit of AMU data collection with the goal of antimicrobial stewardship, these imperfect data should be sought. Viewing AMU and treatment outcome data in an integrated manner over time, as opposed to a single individual comparison of use and outcome, could inform and advance veterinary clinical decision-making and antimicrobial stewardship at the local level in a clinical or production unit, in the context of specific animals and caretakers.

There are individual and corporate examples of treatment outcome data collection in food animal systems or the building of the capacity to do so. VCPR.ORG Veterinary Protocols Manager is a tool for licensed veterinarians to implement treatment protocols with the option of integrating on-farm prescriptions, compliance, and outcomes.¹² The promise of integrated data such as Dairy Brain or the Small Animal Veterinary Surveillance Network would likely allow real-time comparisons of AMU data and animal health outcomes.^13,14

Pen-, barn- or production-unit–level outcomes in livestock systems such as beef, swine, and poultry are often evaluated with mortality or other performance indicators when the production cycle ends or at harvest. Large livestock producers complete their own field trials to measure efficacy in their patient populations before modifying AMU protocols. These types of efficacy assessments may not be generalizable to other populations and may be limited in scope by species, age, region, health status, production type, or other potential factors that influence the outcome of interest. They may also be proprietary in nature and therefore unavailable for extrapolating outside the unit. Consumer-facing certification programs such as Best Aquaculture Practices or the One Health Certified program for farmers and producers include evaluation of high-level outcomes as well.^15,16 Participants in the One Health Program are required to provide detailed and complete antibiotic usage data including number of animals, indication for use, route, dosage, duration, and a crude measure of outcome following treatment. These types of high-level evaluation can still provide red flags for disease incidence and dramatic changes over time that may be compared with AMU data.

Currently available software, data collection tools, and medical terminologies

To compare AMU across veterinary practices, animal production units or farms, and other animal care facilities in a routine and ongoing manner, the tools used to collect data must be able to talk to each other. For example, de Campos and colleagues noted that 3 different records systems were used on the farms from which they collected electronic AMU data, meaning that data could not be easily integrated across the systems.⁴ In the single state in which the study was performed, the dairy farms from which data were collected represented 10% of farms. If this were extrapolated to all farms in all states, the magnitude of the data curation required to combine data from 3 dissimilar systems would be unmanageable. However, there are reasons to be encouraged by available interoperability standards and terminologies.

Current electronic health record (EHR) systems and standards are advancing rapidly, although there is room for improvement.¹⁷ Standardized terminologies for diagnostic and other codes exist, the most comprehensive and annotated being SNOMED-CT Veterinary Extension.¹⁸ Using standardized terminology allows comparisons across entities that agree to use the standards or for which the terminologies are already embedded in EHR or other software. Although there are standards for naming active pharmaceutical ingredients such as the US Adopted Names,¹⁹ rules for trade names of drug products are jurisdiction and company dependent, which can make comparing drug use across jurisdictions, or even within a jurisdiction where multiple formulations of the same active pharmaceutical ingredient are available, very difficult. RxNorm is a drug vocabulary produced by the National Library of Medicine that connects drug codes, nonproprietary names, and trade names for use in EHR systems, but it focuses on human drugs.²⁰ Current AVMA policy on animal health information standards supports standardized health information systems for optimal collection and analysis of information for medical decision-making.²¹

Making data entry easier would likely improve data collection: Feyes et al⁷ describe the development of an electronic prescription form for all teaching hospitals in their system that both increases data captured at the time of the drug order and improves the ability to compare across time. In addition, advances in informatics and big data have made it possible to extract meaningful data using natural language processing on text-based medical records. These techniques have been used to characterize AMU in equine and small animal practice.^22–26 However, the use of natural language-processing and text-mining techniques are not well validated or easy to use, so the need for interoperability still exists. In addition, the need to identify or align standard and nonstandard terminology will also not be eliminated by the use of advanced informatics technologies.

Given that AMR is a One Health issue, ensuring interoperability among veterinary and human health sectors is also important. One effort in that direction is the One Health glossary to support information exchange.²⁷

Lack of technology and systems for recording and extracting AMU data

Record keeping is the basis for all data collection and analysis. The previously cited multicenter studies and a systematic review²⁸ have pointed to the challenges of collecting and summarizing AMU in terms of the systems used for recording AMU. In companion animal practice, records of AMU are most often linked to invoicing and inventory. Individual animal AMU will be captured in a medical record, but the medical record system will define how easily data are extracted. Indication for treatment is often not captured in the same record as the prescription order, so the reason for treatment may not be obvious or easy to extract from a nonstandardized medical record.⁸ In contrast, in food animal production, AMU record keeping is most often centered on drug residue avoidance, although antimicrobial drugs delivered in feed require a Veterinary Feed Directive, which requires reason for treatment. Recording methods have frequently been simple and visual such as hash marks on a hutch or fetlock markers on dairy cows, which could be monitored at the inventory level but will not be tied to individual animal use. Record keeping for these reasons does not create data that can easily be used to analyze AMU and outcome of individual animals.

Even in a single practice, there is rarely a process in place to systematically collect AMU data or data on outcomes of use. A major challenge to capturing these data on a larger scale is the lack of a standardized method or system. This includes standardized metrics as previously discussed, but it also includes data collection protocols and tools, as well as data management and statistical analysis software. Paper records, still used in many areas of practice, are a particular challenge, since they require transcription into an electronic system for analysis or sharing. Another challenge is the accuracy of the medical record. In 1 study, only 64% of problems observed during an office visit were captured in the medical record in small animal primary care practice.²⁹ Whether AMU data are from paper or electronic records, digital literacy and access to broadband in rural areas may be barriers to data collection and analysis.^30,31

An additional concern related to systems of data recording and extraction is the need for confidentiality of animal medical records as well as the desire for anonymity of production systems or data. If large production units contribute data, important differences among units may be inadvertently revealed, such as disease incidence or production parameters that may impact competitiveness or the ability to market animals or product.³²

Absence of widespread common ontology and terminology

Although standardized terminologies exist, they have not been widely adopted. Use of machine learning and text-mining tools is increasing, but they are not yet routinely available and have not been robustly validated across multiple practice types and settings. Data curation in the absence of the ability to mine text or similar tools become onerous and time-consuming, with the result being that it is unlikely to happen.

Abbreviation needs for various software differ, and there are no standards for abbreviation. For example, a glossary is needed to identify the different ways antimicrobial drugs are abbreviated by the manufacturers of antimicrobial susceptibility testing systems in Clinical and Laboratory Standards Institute documents for such testing. Comparing drugs used across institutions in 1 study revealed that there was no standardization, leading to a need for manual curation and translation to make comparisons.⁸ If manual curation is necessary, usefulness for clinical decision-making in real time is hampered.

Finally, any efforts at standardizing terminology require consensus among users, not just about terms and abbreviations but also about what the important outcomes are. We touch on this consideration briefly in the section below on inference making.

Challenges with selection of AMU metrics

There has been and will continue to be discussion about appropriate metrics for characterizing and comparing AMU, which is beyond the scope of this viewpoint. Although different metrics are often correlated, use of one metric may lead to different conclusions than another, and metrics can be selectively used to make a particular point about AMU.^4,33–37In the animal arena, because animal species have different products labeled for their use, dosing regimens, and disease challenges, AMU cannot be compared across sectors.²⁸ In fact, it can be difficult to compare across different types of production systems within a species sector. We argue that this calls for care in comparisons and in conclusions drawn from comparison, rather than eschewing comparisons altogether.

Another important challenge is that the reason AMU is recorded is not usually the same as the reason we are interested in extracting it. Veterinarians and partners most commonly record for the purposes of giving directions for use, collecting payment or invoicing, and residue prevention. Although some parameters can be inferred from recorded data, such as total days of exposure, that is a data curation step that is an added barrier. If invoicing is based on total bottles used or total amount of feed containing a Veterinary Feed Directive drug, the actual regimen administered to individual animals may not be knowable or may be in free text in paper records or the EHR, which hampers the usefulness and interpretation of AMU data.

Caveats to inference making

The ultimate goal of collecting and analyzing AMU data is to draw scientifically valid conclusions and make inferences about the next animal or patient from previous animals or patients. Given an animal patient with a presumptively or definitively diagnosed infectious condition amenable to antimicrobial therapy, based on research information and data from clinical records, which antimicrobial drug would likely provide the greatest probability for a favorable clinical outcome? There are major challenges to inference making, however, including (1) recording the appropriate data in an accurate manner, (2) analyzing the data in a statistically valid and unbiased manner, and (3) drawing appropriate and defensible conclusions from the data.

Record keeping (data recording) is the starting point for inference making about AMU and AMR. In addition to the potential for missing important data such as indication and duration of therapy, animal-level granularity for analyzing AMU and outcomes may not be available in aggregate animal populations such as large production systems. Depending on species, data may only be collected at the population level, most often mortality and close-out data (ie, at the end of the production cycle); even with animals in the tens of thousands, these data may be inadequate to make inferences.

The next step in inference making is to understand data by characterizing them according to factors associated with animal, place, and time. For recording AMU in animals, 2 major components will typically be recording (1) the quantity of antimicrobial drugs administered including dose and duration and (2) the animals receiving the drug, typically on the basis of weight, age, or production class. Analysis of treatment outcomes may be difficult due to limited sample size, lack of time, access to software, and expertise to perform an epidemiologically and statistically appropriate analysis. Addressing these challenges should be an important focus of any system to be developed for collecting and analyzing AMU data. To make the data meaningful and helpful for future clinical decisions, data summaries ought to include the diagnoses or indications for AMU, use according to antimicrobial classes and drugs, doses of antimicrobials administered, durations of therapy, and patient demographics.

In the final step in inference making, drawing conclusions, the analysis of AMU data will often be limited in its ability to provide completely unbiased guidance for medical decisions and case management. The randomized controlled trial is considered as the least biased source of data for comparing antimicrobial drug and regimen selection and estimating the treatment effect, but when unavailable, or of low quality, veterinarians must use the next best available evidence such as data from clinical records.³⁸ Although a critical challenge of data obtained from clinical records is selection bias, Worrall provides arguments that well-designed observational studies can provide reliable scientific evidence for therapeutic outcomes,^39,40 and well-designed observational studies have been shown to provide good estimates of the magnitude of treatment effects.^41,42 The challenge for AMU data collection systems will be to mimic well-designed observational studies in a way that is manageable by the veterinary clinician. These systems must also help veterinarians assess the bias in their data, such as confounding, bias in selection of animals into the study, deviations from the therapeutic plans, missing data, and bias in outcome measurements and selection of the reported results. Other problems include the ecological fallacy, when groups of animals are sampled and analyzed at the group level and the clinician makes inferences at the individual level. The associations between 2 variables at the group (or ecological) level may differ from associations between analogous variables measured at the individual level, resulting in exaggeration of the true association.^43,44 The reverse can also lead to incorrect conclusions based on the atomistic or individualistic fallacy,⁴³ when the veterinarian uses data collected at the individual level to make inferences at the group or population level. An example of this type of situation could be inferring outcomes of antimicrobial therapy from individual dogs for a disease condition and applying that finding at the kennel or veterinary practice level.

Although beyond the scope of this article, veterinarians will need to understand these caveats to making inferences from their collected AMU data, which are nonrandomized observations of outcomes of antimicrobial therapy.

Inadequate funding and resources

Successfully addressing the challenge of AMU data collection requires investment in 3 key arenas, all of which require resources not currently identified: (a) upgrading existing record-keeping systems to allow electronic data collection and export in a manner that integrates with other data sources, (b) encouraging participation from antimicrobial users, and (c) financial support for centralized databases or dashboards.

Given veterinary practice priorities that include the financial health of the practice and workplace efficiencies, veterinary practice owners are not likely to consider investment in upgrading or improving EHR systems to make them interoperable as well as easy to use in the absence of financial returns. Antimicrobial stewardship is a laudable goal but may not be sufficient to incentivize spending resources on software and records, without incurring other benefits. This might be overcome with a combination of positive feedback from clients, accreditation or certification that includes antimicrobial stewardship as a criterion, and software and system upgrades that provide some other benefit as well, but it remains an impediment to collecting AMU data.

The need for antimicrobial users and prescribers to expend time and effort to record and extract AMU data is another hindrance regardless of the type of EHR or other data collection system. In the absence of an automated data collection system that easily integrates with farm or clinic record-keeping systems, the collation, verification, and formatting of data for reporting is a time-consuming endeavor. At present, most animal health record systems have features to record medical information, provide a history of medical treatment, and possibly optimize farm efficiency. They are not designed to identify, collate, and export specific data entries such as AMU. Successful implementation of data collection efforts must consider mechanisms to offset or incentivize the time, effort, and cost. Unlike human medicine, in which interoperability associated with drug prescribing is required for integration with the Centers for Medicare and Medicaid Services to receive reimbursement for medical cost, no such requirements exist on the veterinary side. As a result, there are many different approaches to record management, ranging from handwritten notes to complex software packages. It is unclear what motivators might drive implementation of such software developments that could, on the surface, increase competition between providers. However, it is clear that financial investments will be required, and mechanisms are needed to encourage these software upgrades by providers.

Other participation by antimicrobial users that will require time and effort is the need for data sharing. As with many data sources, detailed AMU data represents intellectual property and potential financial value for companies and individuals. Furthermore, given concerns around data security and privacy, AMU data sharing is not incentivized. The recently released Presidential Advisory Council for Combating Antimicrobial Resistant Bacteria report entitled “Bridging the Gap: Improving Antimicrobial Access and Use Across One Health” concludes that these efforts need to be voluntary and incentivized to be effective.⁴⁵ However, the types of incentives that would be most powerful in encouraging data sharing among veterinarians and veterinary practices remain unknown. Not all incentives need to be financial; for instance, benchmarking reports might provide some incentive for collecting and sharing AMU data. The incentive provided by the previously described One Health Certificate is the product label that demonstrates social responsibility and high animal care standards.

The final resource need is for the development of integrated databases that maintain data security, continually upgrade technologies, and accommodate a constant influx of new data. This requires long-term sustainable funding streams. As new drugs enter the market or new disease entities are recognized, database ontologies will need to be updated. Changing dosage forms and concentrations will also impact reporting and integration of data. While the technologies of artificial intelligence and machine learning hold great promise for further leveraging these data for new insights, these processes are costly from both a computing usage perspective as well as technical expertise. An example of the long-term costs and complexity of collection systems is the proposal for a year-long grant funded by the FDA Center for Veterinary Medicine, which will collect data on a single day in multiple small animal clinics around the US.⁴⁶ The International Consortium of Antimicrobial Stewardship in Agriculture convened by the Foundation for Food and Agriculture recently released a request for applications related to technologies that facilitate antibiotic traceability across the food chain, as well as evaluating the trade-offs and incentives that impact policy and data sharing in this space.⁴⁷