Materials and Methods

Retrospective literature review—Data were collected from all reports of parallel-group clinical RCTs in dogs and cats published from June 2004 to June 2010 in the following journals: American Journal of Veterinary Research, Journal of the American Animal Hospital Association, Journal of the American Veterinary Medical Association, Journal of Small Animal Practice, Journal of Veterinary Emergency and Critical Care, Journal of Veterinary Internal Medicine, Journal of Veterinary Pharmacology and Therapeutics, Veterinary Anaesthesia and Analgesia, Veterinary and Comparative Oncology, Veterinary Dermatology, and Veterinary Surgery. Parallel-group clinical RCTs were defined as studies in which client-owned companion animals were allocated to parallel (uncrossed) groups. Trials were included that evaluated both pharmacological and nonpharmacological interventions, and all studies allocated participants to ≥ 1 treatment groups in addition to a control group (either placebo or active control). Trials that used purpose-bred or colony-group housed animals, historical or literature control groups, or crossover or n-of-1 study designs were excluded. To identify eligible reports, journals were searched by hand and that search was cross-checked with MEDLINE, AGRICOLA, and CAB abstracts.

The following information was extracted from the journal articles: species, number of study subjects, clinical specialty, duration of trial, type and number of trial centers, and number of trial groups. Relevant information on blinding was also collected and included whether authors stated that the study was conducted in a blinded manner, specific descriptions of blinding category (eg, single or double blinded), statements regarding which key trials persons were specifically blinded or not blinded, and whether the method and maintenance of blinding was described. For the purpose of this study, the terms blinded and masked were considered equivalent.

All reported blinded individuals were categorized as one of the following 4 key trial roles: owners of pets, health-care providers (eg, veterinarians or technicians responsible for veterinary care), data collectors (eg, auxiliary staff or veterinarians responsible for measurement, collection, or assessment of outcome data), and data analysts (ie, persons conducting statistical analysis). In addition, for instances in which specific individuals were identified as blinded (eg, investigators or trial personnel) without further description of specific trial duties such as administration of treatments or collection of data, a fifth categorization, ambiguous role, was applied. Trials in which there was no reporting on the blinding status of any specific trial persons were also tabulated. A trial was considered to contain a complete description of the role of all blinded persons if the authors clearly stated who was blinded and what their specific duties were within the study. A trial was considered to contain an incomplete description of the role of all blinded persons if the role of any blinded person was either ambiguous or not reported.

Prospective veterinarian assessment—Veterinarians practicing at the small animal hospital of the University of Pennsylvania were surveyed during 1 week in 2009. Surveys were placed in the veterinarians' hospital mailboxes and returned anonymously to a separate designated mailbox. Participants were asked to identify, from a list of fixed response options, which people involved in single, double, and triple blinded trials were unaware of the treatment group to which the animals were assigned. For each type of trial, respondents could select as many choices as they wished from the following 5 options: owners of enrolled animals, veterinarians, technicians or nurses, data collectors, and statisticians or data analysts. The choices were listed in random order on the survey. Participants also provided information regarding their level of veterinary experience by answering 2 additional questions. The first question asked participants to describe themselves as one of the following: board-certified specialist, residency-trained specialist (not board certified), intern, or resident. The questionnaire did not ask participants to identify their area of specialty, board-certifying body, or other education and training details such as advanced degrees. The second question asked participants to choose from one of the following options to describe how long it had been since their graduation from veterinary school: < 10, 10 to 20, 21 to 30, or 31 to 40 years.

Statistical analysis—Descriptive statistics were calculated. Categorical variables were described as frequencies and percentages.

Results

Retrospective literature review—One hundred ninety-five eligible clinical RCTs were identified in the 11 journals during the specified time interval (Table 1).

There were 16 trials published in 2004, 24 in 2005, 32 in 2006, 34 in 2007, 35 in 2008, 33 in 2009, and 21 in 2010. Eleven clinical specialties were represented: anesthesia and analgesia (n = 76 trials), dermatology (34), internal medicine (23), surgery (16), nutrition (14), cardiology (11), behavior (7), critical care (6), oncology (6), neurology (1), and theriogenology (1). There were 165 trials involving dogs, 29 trials involving cats, and 1 trial of both dogs and cats. One hundred forty-one trials were conducted at a single center (133 academic and 8 private practice), and 54 trials were conducted at multiple centers (7 academic, 27 private practice, and 20 mixed). The median trial duration was 14 days (range, 1 to 2,200 days; mean, 97 days). The median number of enrolled pets per trial was 43 (range, 10 to 426; mean, 67).

One hundred thirty-one (67.2%) studies were conducted in a blinded manner. In 113 of those studies, the authors specifically stated that the study was conducted in a blinded manner, and in 18 of those 131 studies, blinding could be inferred by study design descriptions or statements in the discussion but use of blinding was not specifically stated. Sixty-four (32.8%) studies were not conducted in a blinded manner. Fifty-two (81.2%) of those studies contained no direct references to blinding, and thus, absence of blinding was inferred. In the remaining 12 (18.8%) studies, the authors specifically stated that the trial was open or that key trial personnel were not blinded.

Among the blinded studies in which use of blinding was stated by the authors, 6 (5.3%) were described as single blinded, 49 (43.4%) were described as double blinded, and 58 (52.3%) were simply described as blinded. There were no studies described as triple blinded.

Of the 6 studies described as single blinded, the role of the blinded individuals was categorized as ambiguous in 4 studies, whereas the role of the blinded individuals in the remaining 2 studies was that of data collector.

Of the 49 studies described as double blinded, there was no reporting on the blinding status of any key trial persons in 12 (24.5%) studies, and the role of all blinded individuals was ambiguous in 3 (6.1%) studies. The remaining 34 double-blinded studies explicitly reported on the role of at least 1 blinded individual, with 5 combinations of blinded individuals described. The most common scenario (30 [61.2%] studies) involved blinding owners and ≥ 1 individual with an ambiguous role in the study. Other reported double-blinding scenarios included health-care providers and individuals with ambiguous roles (1 [2.0%]), owners and healthcare providers (1 [2.0%]), data collectors and healthcare providers (1 [2.0%), and health-care providers alone (1 [2.0%]).

Of the 58 studies described as blinded, there was no reporting on the blinding status of any key trial persons in 6 (10.3%) studies, and the role of all blinded individuals was ambiguous in 15 (25.9%) studies. The remaining 37 blinded studies explicitly reported on the role of at least 1 blinded individual, with 8 scenarios of blinded individuals described. The most common scenario (14 [24.1%] studies) was that individuals in the role of data collector or data evaluator were blinded. Other reported blinding scenarios included owners and individuals with an ambiguous role (10 [17.2%] studies), health-care providers alone (5 [8.6%]), health-care providers and data collectors (4 [6.9%]), owners and data collectors (1 [1.7%]), healthcare providers and individuals with an ambiguous role (1 [1.7%]), data analysts and individuals with an ambiguous role (1 [1.7%]), and owners, health-care providers, and individuals with an ambiguous role (1 [1.7%]).

Prospective veterinarian assessment—There were an additional 18 studies containing methodological information that implied blinding (eg, “scoring was performed by…assessors, who were ignorant of the treatment given”)¹³ but without use of any blinding-specific terminology. Of these studies, 6 scenarios of blinded individuals were described: data collectors (8 [44.4%]), individuals with an ambiguous role (4 [22.2%]), health-care providers (2 [11.1%]), health-care providers and data collectors (2 [11.1%]), owners and individuals with an ambiguous role (1 [5.6%]), and owners, health-care providers, and individuals with an ambiguous role (1 [5.6%]).

Of reports for all 131 blinded studies, 42 (32.1%) contained complete descriptions of the role of all blinded persons. Complete descriptions of blinding were included with much lower frequency for studies reported as single or double blinded (5/55 [9.1%]) than for studies that were simply reported as blinded or in which blinding was inferred (39/76 [51.3%]). Overall, pet owners were reported as blinded in 43 (32.8%) studies, health-care providers were reported as blinded in 20 (15.3%), data collectors were reported as blinded in 33 (25.2%), and data analysts were reported as blinded in 1 (0.8%). In 71 (54.2%) studies, the role of at least 1 blinded individual was ambiguous. For 17 (13.0%) studies, there was no explicit information on the blinding status of any individual. There were no studies for which the blinding status of persons in > 2 of the 4 key trial roles was explicitly reported.

Thirty-six questionnaires were distributed, and 21 (58.3%) were returned. Thirteen (6.2%) respondents identified themselves as board-certified specialists, 2 (9.5%) identified themselves as residency-trained specialists, 4 (19.0%) identified themselves as residents, and 2 (9.5%) identified themselves as interns. Twelve (57.1%) respondents had been practicing for < 10 years, including 5 board-certified specialists, 1 residency-trained specialist, and all residents and interns. The remaining 8 board-certified specialists had been practicing for 10 to 20 years (4 respondents), 21 to 30 years (2), and 31 to 40 years (2). The remaining residency-trained specialist had been practicing for 31 to 40 years.

The majority of respondents (66.7%) identified owners of enrolled animals as the only group blinded in a single-blinded study. The remaining 7 participants not only identified owners as blinded, but also selected additional blinded individuals. Three residents identified owners and data collectors as blinded, 3 board-certified specialists and 1 residency-trained specialist identified owners as blinded, and veterinarians and technicians as blinded.

There was no majority opinion regarding which persons were blinded in double-blinded trials. Six combinations of persons were identified, with the most commonly reported combination (47.6%) being veterinarians, technicians, data collectors, and owners. Other answers included data collectors and owners (n = 5 respondents); veterinarians, technicians, and owners (2); veterinarians and owners (2); veterinarians, data collectors, and owners (1); and technicians, data collectors, and owners (1).

The majority of respondents (52.4%) selected all options when asked to describe which persons were blinded in triple-blinded trials. Four other combinations were identified by the remaining respondents: statisticians, technicians, data collectors, and owners (n = 3 respondents); statisticians, data collectors, and owners (3); veterinarians, data collectors, and owners (3); and veterinarians, statisticians, data collectors, and owners (1).

There was no agreement among respondents for any of the questions when responses were stratified by position or years of practice.

Discussion

Of those evaluated in the present study, most companion animal RCTs reported in the recent veterinary literature (2004 to 2010) were conducted in a blinded manner, and a number of nonblinded studies addressed the rationale for their open study design. Overall, 73% of trial reports included statements pertaining to the use of blinding, which compares favorably with previous reviews of RCTs reported in the veterinary literature. These studies, which include laboratory animal,¹ companion animal,^1,4 and livestock and food safety^11,12 RCTs published between the years 1960 and 2008, found that between 4% and 52% of RCT reports contained statements about the use of blinding. This suggests that authors of veterinary RCT reports in companion animal medicine in particular are increasingly recognizing blinding as an essential element of clinical trial design. However, we found that overall clarity of reporting of blinding was poor, particularly in articles where the terms single and double blinded were used. In the present study, most reports of blinding methodology were considered incomplete and there was no consistency in how blinding terminology was used by authors or interpreted by veterinarians. Ambiguous reporting hinders the ability of practitioners to assess the validity of trial results and make informed decisions about applying study findings to their patient populations.

During the course of a trial, several groups of individuals can potentially bias study results through their knowledge of animal allocation to treatment groups. When pet owners or health-care providers are aware of the treatment an animal is receiving, they may be more or less likely to assess a favorable response,⁵ be compliant in treatment administration and study protocols, administer nonstudy treatments, or withdraw patients from a study prematurely.⁶ Similar biases may be introduced when treatment assignments are apparent to persons collecting and interpreting patient data.^6,7 Key trial persons with the potential to introduce bias may include pet owners, health-care providers (eg, veterinarians, technicians, pharmacists, and others), data collectors, persons assessing treatment outcome,⁶ and possibly those involved in data analysis and manuscript preparation.¹⁴

Depending on trial design, blinding of all study personnel is not always possible or necessary.¹⁵ For example, blinding of pet owners and veterinary healthcare providers may not be realistic when comparing a surgical procedure with a medical treatment protocol, but it may still be possible in these circumstances to maintain blinding of the investigators who collect and analyze study data. Conversely, blinding of data collectors may not be important when the primary outcome measure is a purely objective parameter, such as enrolled pet death, but lack of owner or health-care provider blinding could still result in bias associated with compliance and participant withdrawal.¹⁴ Therefore, even in these circumstances, trial reports should clearly state who was blinded and what role or roles each blinded person had in the study.

Unfortunately, the traditional lexicon of blinding does not provide for a clear line of communication between authors and readers with respect to the blinding status of these key trial individuals.^{8–10,14,16,17} A large study⁸ reviewing the use of blinding terminology in human subject clinical trials found that medical textbooks provided 9 definitions of double blinding, and physicians provided 17 interpretations. The most frequently identified definition was chosen by < 40% of physicians and < 50% of textbooks. Our findings showed similar discrepancies in reporting and interpretation.

In the present review, very few studies were described specifically as single blinded, and most did not provide a clear description of who was blinded. Of trials described as double blinded, only 6% provided clear descriptions of who was blinded, and nearly 25% did not provide any description regarding who was blinded. Overall, authors who used the term double blinded reported the blinding status of individuals in 7 ways. Studies described as blinded had similarly inadequate reporting because the role of 1 or all blinded individuals was ambiguous or unreported for 59% of studies. There were no studies reported as triple blinded, although based on published definitions,^8,10 it appeared that some studies might reasonably qualify as triple blinded. Furthermore, studies not described as being blinded were often, in fact, blinded. Seventy reports did not comment on whether the trial was conducted in a blinded manner. Of these studies, > 25% contained methodological descriptions of specific study individuals being blinded but failed to describe the study overall as being blinded. On the basis of previous reports^10,18,19 in the literature, it is possible that some or all studies that did not discuss the presence or absence of blinding (and thus were interpreted as not blinded in the present report) did in fact include blinding in the trial methodology but failed to report it.

Results of our survey also showed notable lack of agreement between veterinarians of various backgrounds when defining and interpreting blinding terminology. Several interpretations were returned for each surveyed term, and the most common interpretation of the term double blinded was selected by less than half of all participants. These findings illustrate the ambiguity of the conventional lexicon of blinding terminology and are in line with published data^8–10 from previous questionnaires associated with human subject clinical trials. Data obtained from our questionnaire were limited by the fact that the responses were from veterinarians only at 1 institution, and those responses may not be generalizable to the greater veterinary community as a whole. A larger distribution of the survey would be necessary to more accurately generalize the discord in veterinarian interpretation of these terms. However, considering the varied nature of backgrounds of the surveyed veterinarians with regard to their institutions of graduate and postgraduate veterinary training, it is very likely that the degree of variability in responses documented in this group would also be documented in a larger study including multiple institutions.

The CONSORT Group has developed evidence-based guidelines for the reporting of human subject RCTs. These guidelines are provided as a tool to promote transparent reporting and improve the ability of clinicians to interpret trial data that may impact future patient care. The statement's recommendations with regard to blinding are that authors abandon using the terms single and double blinded and instead explicitly state who is blinded and what responsibilities each blinded person or group had during the course of the study.^20,21 The CONSORT guidelines are endorsed by hundreds of scientific journals, including 3 veterinary publications (Journal of Veterinary Internal Medicine, Preventative Veterinary Medicine, and The Veterinary Journal). The only published systematic review²² evaluating the impact of the CONSORT statement found overall improved reporting of RCTs in journals that endorsed CONSORT guidelines, compared with nonendorsing journals. However, when the impact on reporting of blinding was specifically examined, it did not appear that articles published in CONSORT-endorsing journals were more likely to report explicitly on the blinding status of key trial persons, prompting speculation that journals will have to enforce the statement, rather than simply endorse it, if any effective improvement in reporting of blinding is to occur.

Since the original CONSORT statement was published in 1996, many similar initiatives have been developed to address reporting and methodological issues within specific fields of biomedical research. The ARRIVE guidelines²³ and REFLECT statement²⁴ are recently published CONSORT-based recommendations that seek to address methodological issues specific to veterinary research and to improve reporting in veterinary studies. The ARRIVE and REFLECT initiatives pertain specifically to trials that use laboratory animals and livestock study subjects, respectively. Both initiatives have been reprinted, endorsed, or recommended by authors in numerous scientific and veterinary journals. To date, there are no published data regarding the impact of these recommendations on the quality of subsequent veterinary research; however, this is to be expected given the relatively short time since their publication.

The ARRIVE guidelines advocate for the use of blinding as a means to reduce bias, but they provide limited direction for authors in this regard by simply stating, “if done, describe who was blinded and when.”²² The inconsistencies we have identified regarding what constitutes a complete and appropriate description of blinding in veterinary RCTs suggest that authors would benefit from more definitive guidelines with respect to the reporting of blinding. The REFLECT statement achieves this by way of a separate Explanation and Elaboration companion document²⁵ containing a comprehensive discussion of specific recommendations with respect to blinding. Like the CONSORT statement, REFLECT encourages authors to explicitly describe the process of blinding, state who was blinded and their role in the study, and provide an explanation in the event that blinding is not used. Furthermore, the REFLECT initiative makes the salient point that subjects of veterinary studies cannot themselves be blinded and proposes a novel 3-tier system categorizing various study personnel who might be blinded or not blinded. However, REFLECT does permit use of the terms single, double, and triple blinded, despite acknowledging the ambiguity of such language. The intention is clearly that authors following the REFLECT recommendations would use these terms only in conjunction with explicit descriptions of blinded personnel and their role in the study, but the new data in the present study support unqualified rejection of this terminology. Continuing to sanction even provisional use of these terms has the potential to propagate the very ambiguity these valuable reporting guidelines seek to eliminate.

The data of the present study highlight the shortcomings of commonly used blinding terminology in the veterinary literature. For most trials evaluated, reporting of blinding was incomplete, ambiguous, or absent. There was no consistency in how the terms single, double, and triple blinded were used by authors or interpreted by veterinarians. Studies containing the terms single and double blinded were also unlikely to clearly state the role of blinded individuals. Our data support the recommendation that authors and editors of veterinary journals should abandon vague descriptions of blinding and that reporting guidelines for veterinary RCTs should unambiguously reject traditional blinding terminology. Authors should explicitly report on the role and blinding status of all key trial personnel as suggested in the CONSORT and REFLECT statements. It is only through clear reporting of clinical trial methodology that veterinarians can use the results of these pivotal studies to guide appropriate veterinary patient care.