Background

In human research, use of interim analyses during RCTs at predetermined intervals has increased.² The results of these analyses are used to make sample size adjustments, change the random allocation of subjects between study groups, or terminate a trial.^3,4 A priori–specified rules that are used to make decisions on discontinuing RCTs are termed stopping rules.⁵ Stopping rules are applicable when enrollment of subjects or continued application of the intervention can feasibly be stopped after the accumulation of sufficient outcome data and prior to attainment of the planned sample size. Such rules are used in 3 situations. Negative stopping refers to stopping RCTs because of adverse effects of a new intervention or because the intervention yields significantly worse results than would the standard treatment. Positive stopping is used when the benefits of the new treatment are significantly greater than those of a placebo or standard treatment. Futility stopping is used when the results are unlikely to change with continuation of the trial.⁵

Stopping rules most commonly involve use of conservative P values that vary in relation to the time at which the interim analysis is performed in an RCT. When the P value is equal to or less than a prespecified value when the analysis is performed, the trial is stopped.³ Considerations when planning interim analysis and specifying stopping rules include which outcomes to analyze, the number of interim analyses to include, and when to perform the analyses.⁶ The primary objective of stopping rules is to ensure that the maximum number of patients receive the best possible medical care, while maintaining the validity of the RCT.⁷

Advantages of Stopping Rules

The primary objective of stopping rules is to protect study subjects from receiving a treatment that has negative impacts or from not receiving a treatment that would have a substantially better impact than the alternative. There are many advantages to applying these rules. For instance, early stoppage of RCTs may help conserve financial, human, and animal resources or redirect these resources to other projects. Interim analyses may be particularly useful when there is a high degree of uncertainty associated with the assumptions for sample size calculation. Stopping rules may hasten the delivery of an effective intervention to clinical practice by decreasing the time required to complete an RCT. Primarily, however, researchers conducting RCTs are compelled by ethical principles that are designed to protect the individual rights of the patients while adhering to an obligation to deliver valid scientific information.⁵

Equipoise is an ethical requirement for RCTs that involves use of interventions with uncertain impacts. It has been defined as the “genuine uncertainty on the part of the clinical investigator,”⁸ which makes it ethical to pursue a particular RCT. When this uncertainty no longer exists, investigators are obligated to administer the best treatment to all patients. Interim analyses can be used to assess whether equipoise still exists, and stopping rules attempt to specify the statistical degree of certainty that should result in the loss of an investigator's ethical right to continue enrollment of subjects into the study. For veterinarians, these individual ethical considerations are compounded when applying interventions with uncertain effects to privately owned animals. Not only do veterinarians have an obligation to relieve animal suffering, they also have an obligation to owners to provide the best possible medical care to their patients. As suggested by Michael Fox, “Veterinary medical ethics essentially mandates that the best scientific knowledge and medical and surgical expertise be provided to animals on their owner's request and payment for professional services.”⁹ In RCTs in which privately owned animals are used, the obligations to owners and their animals must be weighed against the researcher's obligation to deliver valid scientific information to the medical community.

Disadvantages of Stopping Rules

Interim analyses must be conducted with caution. Frequent examination of the data increases the likelihood that significant results will be found. Because unplanned interim analyses inflate the likelihood of making a type I error, the performance of unplanned interim analyses must be reported with the results of the trial. Although modern statistical methods allow preplanned interim analyses of data while preserving the probability of a type I error,⁶ RCTs stopped early because an apparent benefit is detected can result in reporting of implausibly large treatment benefits.² Such misleading claims often originate from RCTs in which few outcome events have accumulated. Montori et al² identified 143 human RCTs stopped for apparent benefit. Of these RCTs, trials with fewer than the median number of events were 28 times as likely to yield a treatment effect higher than the median effect for all trials. Although study subjects are entitled to the best possible medical care, society must be shielded from “overzealous premature claims of treatment benefit.”¹⁰ Alternatively, other authors have concluded that RCTs stopped early only mildly overestimate the treatment effect and are unlikely to yield an incorrect conclusion.¹¹ In 1 review,¹² only 1 of 18 cancer RCTs stopped early for benefit led to a different conclusion than was reached after later follow-up work.

Other potential disadvantages of the use of stopping rules include misuse to conserve resources or garner media attention. Additionally, when an RCT is stopped and the research question appears to have been answered, it may be more difficult for researchers to win funding to repeat or expound upon the results. Important secondary outcomes may not be able to be analyzed when too few events have accumulated in the trial. When defining stopping rules, researchers and data safety and monitoring boards must balance the rights of the study subjects against the probability of reporting biased treatment effects.

Stopping Rules as a Tool for Client Communication

Informed consent in veterinary medicine, based on the principle of beneficence, was modeled after the legal doctrine in the human medical community,¹³ Just as a human surgeon who performs surgery without a patient's consent commits assault, a veterinarian who uses client-owned animals in clinical research without the owner's knowledge has violated an important ethical principle.¹³ In veterinary research, informed consent is not derived directly from the study subjects. Instead, the process is intended to give animal owners sufficient information to make a decision about whether their animals should participate. Also unlike human research, study subjects in veterinary research do not have the ability to remove themselves from a trial. Although owners may be free to withdraw their animals from a trial, they are not routinely provided with the results of any interim analyses to consider such a decision. Release of interim results to clinicians or patients may disrupt a clinical trial.⁴ However, others have described the policy of withholding these data from patients as unethical and fraudulent.¹⁴ Routine disclosure of interim results to owners who lack the scientific background to interpret these data would likely result in unacceptably high dropout rates. However, careful explanations of planned interim analyses and stopping rules may complement the informed consent process.

Investigators in companion animal RCTs are faced with the challenge of communicating with a large number of clients about the risks and benefits of their pets' participation. The human-animal bond and the expectations of pet owners for high-quality medical care may both affect the ability of investigators to recruit study subjects and retain them for the entire follow-up period. In companion animal trials, stopping rules may assure owners at enrollment that their pets will not unnecessarily receive a detrimental intervention or be deprived of a promising new one.

Randomized controlled trials involving food-producing species are increasingly performed on commercial farms with privately owned animals.¹⁵ Stopping rules may serve to protect farm owners from economic losses incurred by adverse effects of new treatments or the opportunity costs of not using an effective intervention on the entire herd. Farm owners may be more likely to consent to an RCT conducted on their farm when assurances can be given that the trial will be stopped if results demonstrate that farm production is substantially reduced by the intervention. As stated by David Granstrom, “Knowledge of the purpose, progress, and impact of the research helps relieve the tension created by competing demands for optimal production and research results.”¹⁶ The strictness of the stopping rules may need to be influenced not only by magnitude of the probability of a type I error but also by the farm owner's tolerance for potential costs, lest investigators risk not obtaining the consent of the ownership to perform the trial. For livestock clinical trials with multiple outcomes such as weight gain and milk production, investigators must consider which outcomes to include in the interim analysis and which outcome to weight more heavily if the results are discordant. Inclusion of stopping rules in the design of production animal RCTs could be useful to assure producers about the impacts of research on production, while maintaining the validity of the trial.

Frequency of Stopping-Rule Use in Veterinary RCTs

Randomized controlled trials are important for providing unbiased information to facilitate clinician decision making. However, reports of RCTs in human and veterinary medicine often lack components necessary to judge the validity of the data.^17,18 The CONSORT statement¹⁹ was developed by an international group of statisticians and clinical trial experts to improve the transparency, completeness, and clarity in reporting of human RCTs. Many of the 22 items in the statement are listed because “empirical evidence indicates that no reporting of this information is associated with biased estimates of treatment effect, or because the information is essential to judge the reliability of the data.”¹⁹ Prospectively defined stopping rules are included on the list for trials in which their use would be warranted. In 2003, the Australian Veterinary Journal called for adoption of reporting standards outlined in the CONSORT statement.²⁰ The statement was recently modified to create the REFLECT statement,²¹ which provides guidelines for RCTs in livestock and food safety. The authors of the REFLECT statement retained stopping rules on the list and pointed out that “if an intervention is particularly efficacious, or if it causes harm, it may be ethically appropriate to end the trial.”²¹ In 1987, Geller and Pocock⁶ stated, “With current statistical methods, it is now recommended that planned interim analyses be included in any trial protocol.”

Even so, the use of stopping rules in the veterinary RCTs is rare. A review of 100 reports of RCTs involving livestock and food safety revealed that none mentioned the a priori establishment of stopping rules.²² This review involved examination of only those RCTs that included livestock or food safety outcomes. To further assess the frequency of stopping-rule use in veterinary RCTs, we selected and reviewed 30 reports of RCTs published between January 2000 and October 2009 from the Journal of the American Veterinary Medical Association, American Journal of Veterinary Research, Journal of Dairy Science, Journal of Veterinary Internal Medicine, and Equine Veterinary Journal. These journals were specifically chosen in order that RCTs from multiple companion animal and food animal species could be evaluated. Manuscripts in which the use of stopping rules would be appropriate were selected. Selected manuscripts described studies in which privately owned animals were used; had illness, death, or a production metric as an outcome measurement; and had a period of enrollment that was of sufficient duration such that implementation of stopping rules would have been feasible. The number of RCTs reviewed was calculated on the basis of a 95% confidence for detection, assuming that stopping rules were reported in 10% of selected RCTs. Of the RCT reports reviewed, only 1 report²² described the use of an interim analysis, and none described the use of stopping rules that had been defined before the study began.

Conclusion

Use of stopping rules in veterinary RCTs is met with several challenges. Often, a mechanism for planned interim analysis by data monitoring boards does not exist. Funds may not be available for performance of interim analysis by an independent third party. The use of independent monitoring committees may appear cumbersome and unnecessary for RCTs with a short duration or few study subjects. Lastly, the ethical consequences of not performing interim analyses in RCTs involving nonhuman subjects may not appear as grave as when human subjects are involved. Even so, stopping rules have been listed as an important component of reporting of RCTs in veterinary medicine.²² Although modern statistical methods enable interim analyses without inflating the likelihood of committing a type I error, the use of stopping rules remains controversial and has been associated with biased treatment effects.¹ This does not, however, obviate the ethical obligations of veterinary researchers to the owners and to their animals. Whether in companion animal or agricultural settings, stopping rules could be used to alleviate animal welfare concerns, provide assurances to animal owners that foster research, and improve adherence to the principles of veterinary research ethics.