Precision and accuracy of clinical illness scores, compared with pulmonary consolidation scores, in Holstein calves with experimentally induced Mycoplasma bovis pneumonia

David E. Amrine Department of Diagnostic Medicine and Pathobiology, College of Veterinary Medicine, Kansas State University, Manhattan, KS 66506.

Search for other papers by David E. Amrine in
Current site
Google Scholar
PubMed
Close
 DVM
,
Brad J. White Department of Clinical Sciences, College of Veterinary Medicine, Kansas State University, Manhattan, KS 66506.

Search for other papers by Brad J. White in
Current site
Google Scholar
PubMed
Close
 DVM, MS
,
Robert Larson Department of Clinical Sciences, College of Veterinary Medicine, Kansas State University, Manhattan, KS 66506.

Search for other papers by Robert Larson in
Current site
Google Scholar
PubMed
Close
 DVM, PhD
,
David E. Anderson Department of Clinical Sciences, College of Veterinary Medicine, Kansas State University, Manhattan, KS 66506.

Search for other papers by David E. Anderson in
Current site
Google Scholar
PubMed
Close
 DVM, MS
,
Derek A. Mosier Department of Diagnostic Medicine and Pathobiology, College of Veterinary Medicine, Kansas State University, Manhattan, KS 66506.

Search for other papers by Derek A. Mosier in
Current site
Google Scholar
PubMed
Close
 DVM, PhD
, and
Natalia Cernicchiaro Department of Diagnostic Medicine and Pathobiology, College of Veterinary Medicine, Kansas State University, Manhattan, KS 66506.

Search for other papers by Natalia Cernicchiaro in
Current site
Google Scholar
PubMed
Close
 DVM, PhD

Abstract

Objective—To determine the precision of a clinical illness score (CIS) system for identification of clinical signs in calves with experimentally induced Mycoplasma bovis pneumonia and to evaluate the accuracy of CISs in relation to pulmonary consolidation scores assigned at necropsy.

Animals—178 Holstein bull calves that were 52 to 91 days of age at the time of pneumonia induction.

Procedures—5 trials involved calves challenged with M bovis and scheduled for euthanasia and necropsy 12 to 24 days afterward. Nine veterinarian observers with various degrees of experience simultaneously assigned CISs to calves within 48 hours before necropsy. The precision of the CIS system among observers was evaluated via the Cohen κ statistic. The accuracy of each observer's CISs relative to 6 cutoffs (≥ 5%, ≥ 10%, ≥ 15%, ≥ 20%, ≥ 25%, and ≥ 30%) of percentage pulmonary consolidation was determined by comparing prenecropsy CISs with the gross pulmonary consolidation scores assigned at necropsy. Estimates for sensitivity and specificity were calculated relative to the 6 pulmonary consolidation cutoffs.

Results—A slight level of agreement was evident among observers (κ range, 0.10 to 0.21 for the individual trials) and overall (κ = 0.16; 95% confidence interval, 0.10 to 0.24). Median sensitivity and specificity changed with pulmonary consolidation score cutoff. Median sensitivity for all observers ranged from 81.7% to 98.9%, and median specificity ranged from 80.8% to 94.9% over all cutoff values.

Conclusions and Clinical Relevance—Agreement among observers assigning CISs to calves was low; the accuracy of the CIS system in relation to that of pulmonary consolidation scoring varied with the severity of consolidation considered to represent bovine respiratory disease.

Abstract

Objective—To determine the precision of a clinical illness score (CIS) system for identification of clinical signs in calves with experimentally induced Mycoplasma bovis pneumonia and to evaluate the accuracy of CISs in relation to pulmonary consolidation scores assigned at necropsy.

Animals—178 Holstein bull calves that were 52 to 91 days of age at the time of pneumonia induction.

Procedures—5 trials involved calves challenged with M bovis and scheduled for euthanasia and necropsy 12 to 24 days afterward. Nine veterinarian observers with various degrees of experience simultaneously assigned CISs to calves within 48 hours before necropsy. The precision of the CIS system among observers was evaluated via the Cohen κ statistic. The accuracy of each observer's CISs relative to 6 cutoffs (≥ 5%, ≥ 10%, ≥ 15%, ≥ 20%, ≥ 25%, and ≥ 30%) of percentage pulmonary consolidation was determined by comparing prenecropsy CISs with the gross pulmonary consolidation scores assigned at necropsy. Estimates for sensitivity and specificity were calculated relative to the 6 pulmonary consolidation cutoffs.

Results—A slight level of agreement was evident among observers (κ range, 0.10 to 0.21 for the individual trials) and overall (κ = 0.16; 95% confidence interval, 0.10 to 0.24). Median sensitivity and specificity changed with pulmonary consolidation score cutoff. Median sensitivity for all observers ranged from 81.7% to 98.9%, and median specificity ranged from 80.8% to 94.9% over all cutoff values.

Conclusions and Clinical Relevance—Agreement among observers assigning CISs to calves was low; the accuracy of the CIS system in relation to that of pulmonary consolidation scoring varied with the severity of consolidation considered to represent bovine respiratory disease.

Contributor Notes

This manuscript represents a portion of a dissertation submitted by the senior author to the Kansas State University Department of Diagnostic Medicine and Pathobiology as partial fulfillment of the requirements for a Doctor of Philosophy degree.

Supported in part by CEVA/Biomune.

Presented in abstract form at the Phi Zeta Day of Kansas State University, Manhattan, Kan, March 2012.

The authors thank Drs. Matt Miesner, Brandon Fraser, Carrie Wheeler, and Amanda Hartnack for assistance in data collection.

Address correspondence to Dr. White (bwhite@vet.k-state.edu).
  • 1. Apley M. Bovine respiratory disease: pathogenesis, clinical signs, and treatment in lightweight calves. Vet Clin North Am Food Anim Pract 2006; 22: 399411.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2. Nicholas RA. Mycoplasma bovis: disease, diagnosis, and control. Res Vet Sci 2003; 4: 105112.

  • 3. Wittum TE, Woollen NE, Perino LJ, et al. Relationships among treatment for respiratory tract disease, pulmonary lesions evident at slaughter, and rate of weight gain in feedlot cattle. J Am Vet Med Assoc 1996; 209: 814818.

    • Search Google Scholar
    • Export Citation
  • 4. Gardner BA, Dolezal HG, Bryant LK, et al. Health of finishing steers: effects on performance, carcass traits, and meat tenderness. J Anim Sci 1999; 77: 31683175.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 5. White BJ, Renter DG. Bayesian estimation of the performance of using clinical observations and harvest lung lesions for diagnosing bovine respiratory disease in post-weaned beef calves. J Vet Lab Diagn 2009; 21: 446453.

    • Search Google Scholar
    • Export Citation
  • 6. Hayes G, Mathews K, Kruth S, et al. Illness severity scores in veterinary medicine: what can we learn? J Vet Intern Med 2010; 24: 457466.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7. Dohoo I, Martin W, Stryhm H. Veterinary epidemiologic research. 2nd ed. Charlottetown, PE, Canada: VER Inc, 2009; 5: 9294.

  • 8. White BJ, Anderson DE, Renter DG, et al. Clinical, behavioral, and pulmonary changes following Mycoplasma bovis challenge in calves. Am J Vet Res 2012; 73: 490497.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 9. Perino LJ, Apley M. Clinical trial design in feedlots. Vet Clin North Am Food Anim Pract 1998; 14: 243266.

  • 10. Fajt VR, Apley MD, Roth JA, et al. The effects of danofloxacin and tilmicosin on neutrophil function and lung consolidation in beef heifer calves with induced Pasteurella (Mannheimia) haemolytica pneumonia. J Vet Pharmacol Ther 2003; 26: 173179.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159174.

  • 12. Kristensen E, Dueholm L, Vink D, et al. Within- and across-person uniformity of body condition scoring in Danish Holstein cattle. J Dairy Sci 2006; 89: 37213728.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13. Wagai J, Senga J, Fegan G, et al. Examining agreement between clinicians when assessing sick children. PloS One [serial online] 2009; 4:e4626. Available at www.plosone.org/article/info%3adoi%2F10journal.pone.0004626. Accessed Jan 21, 2012.

    • Search Google Scholar
    • Export Citation
  • 14. March S, Brinkmann J, Winkler C. Effect of training on the inter-observer reliability of lameness scoring in dairy cattle. Anim Welf 2007; 16: 131133.

    • Search Google Scholar
    • Export Citation
  • 15. Keegan KG, Dent EV, Wilson DA, et al. Repeatability of subjective evaluation of lameness in horses. Equine Vet J 2010; 42: 9297.

  • 16. Channon AJ, Walker AM, Pfau T, et al. Variability of Manson and Leaver locomotion scores assigned to dairy cows by different observers. Vet Rec 2009; 164: 388392.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17. Thompson PN, Stone A, Schultheiss WA. Use of treatment records and lung lesion scoring to estimate the effect of respiratory disease on growth during early and late finishing periods in South African feedlot cattle. J Anim Sci 2006; 84: 488498.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18. Schneider MJ, Tait RG Jr, Busby WD, et al. An evaluation of bovine respiratory disease complex in feedlot cattle: impact on performance and carcass traits using treatment records and lung lesion scores. J Anim Sci 2009; 87: 18211827.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19. Bryant LK, Perino LJ, Griffin D, et al. A method for recording pulmonary lesions of beef calves at slaughter, and the association of lesions with average daily gain. Bovine Pract 1999; 33: 163173.

    • Search Google Scholar
    • Export Citation

Advertisement