Materials and Methods

Procedures—Records from a previous study⁷ of 144 cows from 6 dairies in which ACM had been diagnosed were reviewed for information on signalment, physical examination findings, results of bacterial culture of blood samples, and outcome. All cows had been vaccinated with a bacterin containing the J-5 strain of Escherichia coli.^a Cows had been examined at the time ACM had initially been identified (time 0), and rectal temperature, heart rate, rumen contraction rate, hydration, and attitude had been recorded, along with physical characteristics of the affected mammary glands (firmness and swelling) and viscosity of the secretions. Information on voluntary feed intake and milk production was not available; however, most cows had a decrease in milk production.

Severity scoring systems—The 4 severity scoring systems that were compared included the SSS, LSS, LS1, and LS2 systems. The SSS was based on systemic disease signs at the time ACM was identified (ie, time 0) and allowed cows to be classified as having mild, moderate, or severe disease (Appendix 1). The LSS also allowed cows to be classified as having mild, moderate, or severe disease, but was based only on local disease signs (Appendix 2).

The LS1 and LS2 were both based, with minor variations, on the International Dairy Federation definitions of mastitis severity.⁶ An LS1 of mild was assigned if the milk was grossly abnormal but the affected gland was grossly normal and there were no systemic signs of disease; an LS1 of moderate was assigned if the milk was grossly abnormal and the affected gland was inflamed but there were no systemic signs of disease; and an LS1 of severe was assigned if the milk was grossly abnormal, the affected gland was inflamed, and there were systemic signs of disease. Systemic signs of disease were defined as a rectal temperature > 39.4°C (102.9°F), rumen contraction rate of 0 contractions/min, and hydration score > 1.

An LS2 of mild was assigned if the milk was grossly abnormal and no other local or systemic signs of inflammatory disease were seen; an LS2 of moderate was assigned if the milk was grossly abnormal and there was firmness or swelling of the affected mammary gland, but no systemic signs of inflammatory disease were seen; and an LS2 of severe was assigned if the milk was grossly abnormal, there was firmness or swelling of the affected mammary gland, and at least 2 of the following systemic disease signs were seen: rectal temperature ≥ 39.5°C (103°F), hydration score of 2 or 3 (moderate to marked enophthalmos), and attitude score of 2 (signs of marked depression).

Outcomes—Cows were classified as bacteremic if bacterial culture of blood samples yielded Escherichia coli, Klebsiella pneumoniae, Mannheimia hemolytica, Pasteurella multocida, Enterobacter agglomerans, or Salmonella Typhimurium.⁸ Cows that died during the episode of ACM or that were culled within 30 days as a direct result of the episode of ACM (eg, because of mastitis or low production) were classified as nonsurvivors. All others were classified as survivors.

Statistical analysis—The χ² test was used to compare distributions of severity classifications (mild, moderate, and severe) among the 4 scoring systems. In addition, χ² tests were used to compare among scoring systems the proportions of cows classified as having severe disease that were subsequently found to have bacteremia or to have died or been culled. For each scoring system, sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated separately for the ability to predict the presence of bacteremia (yes vs no) and ability to predict outcome (survived vs died or culled). For these calculations, a scoring classification of severe was considered a positive test result. To compare scoring systems, the true-positive rate (sensitivity) for each system was plotted as a function of the false-positive rate (1–specificity), and the scoring system closest to the top left corner of the plot (ie, a true-positive rate of 1 and a false-positive rate of 0) was considered the best system for identifying the outcome of interest.

For the LS1 and LS2 systems, all calculations were repeated with high heart rate (ie, heart rate > 90 beats/min) considered as an additional potential systemic disease sign. All analyses were performed with standard software.^b For all analyses, values of P < 0.05 were considered significant.

Results

Of the 144 cows with ACM included in the previous study,⁷ 99 were included in the present study. The remaining 45 cows were excluded from the present study because information on 1 or more variables was missing. Of the 99 cows in the study, 12 had bacteremia and 20 died or were culled.

With the SSS, LSS, LS1, and LS2 systems, 21%, 53%, 63%, and 38% of the cows, respectively, were classified as having severe disease (Table 1). The percentage of cows classified as having severe disease was significantly (P = 0.013) higher for the LSS, LS1, and LS2 systems than for the SSS system.

The percentage of cows classified with the SSS system as having severe disease that had bacteremia (7/21 [33%]) was significantly (P = 0.025) higher than the percentage of cows classified with the LSS system as having severe disease that had bacteremia (6/53 [11%]; Table 2). Similarly, the percentage of cows classified with the SSS system as having severe disease that died or were culled (10/21 [48%]) was significantly (P = 0.012) higher than the percentage of cows classified with the LSS system as having severe disease that died or were culled (10/53 [19%]).

With the LS1 system, all 12 cows with bacteremia were classified as having severe disease (ie, sensitivity = 1), and 18 of the 20 cows that died or were culled were classified as having severe disease (ie, sensitivity = 0.9; Table 3). However, specificity of the LS1 system was low, primarily because 63 of the 99 cows were classified as having severe disease. Specificity of the SSS system was higher than specificity of the other systems. Although the LS2 system had higher sensitivity than the SSS system, it had lower specificity. For all scoring systems, the positive predictive value was low because of the low prevalence of bacteremia and death or culling. Conversely, the negative predictive value was high for all scoring systems. Accuracy was highest for the SSS system.

Examination of the scatterplot for true-positive rate versus false-positive rate indicated that the SSS and LS2 systems were similar in their ability to correctly identify cows with bacteremia or an adverse outcome (ie, death or culling; Figure 1).

View Full Size

Scatterplot of the true positive rate (sensitivity) versus the false-positive rate (1–specificity) for 4 disease severity scoring systems used to predict the presence of bacteremia or an adverse outcome (died or were culled within 30 days) for cows with ACM.

Citation: Journal of the American Veterinary Medical Association 229, 2; 10.2460/javma.229.2.259

• Download Figure
• Download figure as PowerPoint slide

Tachycardia (heart rate > 90 beats/min) was observed in 54 of the 99 cows in the study. Inclusion of tachycardia as a potential systemic sign of disease resulted in a shift of 13 cows from the moderate to the severe group with the LS1 system and a shift of 12 cows from the moderate to the severe group with the LS2 system. However, none of these cows had bacteremia or died or were culled.

Discussion

Results of the present study suggest that a scoring system for disease severity in cows with ACM based on systemic disease signs (ie, the SSS system) was better for identifying cows with bacteremia or an adverse outcome (ie, death and culling) than was a system based on local inflammatory changes of the mammary gland (ie, the LSS system). The 2 scoring systems that were based on a combination of local and systemic disease signs (ie, the LS1 and LS2 systems) were intermediate in their discriminatory abilities, although the system that emphasized graded evaluations of systemic disease signs (ie, the LS2 system) performed better and nearly as well as the SSS system.

A clinically useful severity scoring system would correctly identify only those cows that require specific management to achieve a desirable outcome. For example, systemic antimicrobial treatment has been advocated for cows with clinical mastitis at greater risk of developing bacteremia.^5,8 To be clinically useful, therefore, a severity scoring system should not only have adequate sensitivity to identify most cows at risk for bacteremia (and thus requiring antimicrobial treatment) but also have adequate specificity to avoid the expense of treating cows not likely to be bacteremic. The LSS and LS1 systems had sensitivities of 0.50 and 1.0, respectively, for identifying bacteremia. However, both of these systems had low specificities because with these systems, 53% and 63%, respectively, of all cows were classified as having severe disease when, in fact, only 12% of cows had bacteremia. Thus, if the LS1 system had been used to determine which cows would receive systemic antimicrobial treatment, all 12 cows with bacteremia would have been treated, but an additional 51 that did not have bacteremia would also have been treated unnecessarily. If the LSS system had been used for the same purpose, only 6 of the cows with bacteremia would have been treated and 47 additional cows without bacteremia would have received unnecessary treatment. In contrast, if the SSS system had been used, 7 of the cows with bacteremia and only 14 cows without bacteremia would have been treated with antimicrobials.

In the present study, the SSS and LS2 systems were fairly similar in their ability to accurately identify (ie, classify as having severe disease) cows at risk for not surviving (ie, dying during the mastitis episode or being culled within 30 days after the episode). Use of a disease severity scoring system that accurately identifies cows at risk for an adverse outcome would allow more judicious use of resources when making management and treatment decisions. For example, 10 of 21 (48%) cows classified as having severe disease when the SSS system was used died or were culled, whereas only 10 of the 53 (19%) cows classified as having severe disease with the LSS system had this outcome.

The low prevalence of bacteremia (12%) and death and culling (20%) in the study population resulted in low positive predictive values for all scoring systems that were evaluated, although the SSS system had the highest positive predictive values. Accuracy was highest for the SSS system, which was based on systemic signs of disease alone, and the LS2 system, which included graded evaluations of systemic disease severity, and lowest for the LS1 system, which did not include graded evaluations of systemic disease severity, and the LSS system, which included only local signs of inflammation.

In the present study, we used a scatterplot to compare performance of the various disease severity scoring systems. This evaluation indicated that systems that incorporated graded evaluations of systemic disease severity (SSS and LS2) more accurately identified animals likely to be bacteremic or have adverse outcomes. Thus, these systems may provide guidance for more prudent management and treatment decisions than those based primarily on local inflammatory changes of the mammary gland (LSS and LS1).

The scoring system from which the LS2 system used in the present study was derived⁵ incorporated poorly defined signs such as “shock” and “decreased milk production.” For instance, although shock was defined as high heart rate, tenting of the eyelids, shivering, and slow capillary refill time, it was unclear whether all or only some of these signs needed to be present for a cow to be classified as having “shock.” Similarly, a cutoff for defining “decreased milk production” was not given. For this reason, we elected to modify the previous scoring system for use in the present study.

Two previously described mastitis severity scoring systems^3,5 included heart rate as a systemic disease sign and considered a heart rate > 90 beats/min as indicative of more severe clinical mastitis. A heart rate > 90 beats/min was common among cows included in the present study; however, we did not include heart rate in the LS1 and LS2 systems because it can be affected by ambient temperature and humidity and by fear and activity levels. When heart rate was included as a factor in these scoring systems, there was a shift of 13 (LS1 system) or 12 (LS2 system) cows from the moderate to the severe group, but because none of these cows had bacteremia and none died or were culled, including heart rate served to decrease performance of the LS1 and LS2 systems. That said, modification of these scoring systems to include tachycardia, but with a higher cutoff value for heart rate, might still prove beneficial.