Use of a multivariable model to estimate the probability of discharge in hospitalized foals that are 7 days of age or less

Barton W. RohrbachDepartment of Large Animal Clinical Sciences, College of Veterinary Medicine, University of Tennessee, Knoxville, TN 37996.

Search for other papers by Barton W. Rohrbach in
Current site
Google Scholar
PubMed
Close
 VMD, MPH, DACVPM
,
Benjamin R. BuchananDepartment of Large Animal Clinical Sciences, College of Veterinary Medicine, University of Tennessee, Knoxville, TN 37996.

Search for other papers by Benjamin R. Buchanan in
Current site
Google Scholar
PubMed
Close
 DVM, DACVIM
,
Jennifer M. DrakeDepartment of Large Animal Clinical Sciences, College of Veterinary Medicine, University of Tennessee, Knoxville, TN 37996.

Search for other papers by Jennifer M. Drake in
Current site
Google Scholar
PubMed
Close
 BS
,
Frank M. AndrewsDepartment of Large Animal Clinical Sciences, College of Veterinary Medicine, University of Tennessee, Knoxville, TN 37996.

Search for other papers by Frank M. Andrews in
Current site
Google Scholar
PubMed
Close
 DVM, MS, DACVIM
,
Fairfield T. BainHagyard Equine Medical Institute, 4250 Iron Works Pike, Lexington, KY 40511.

Search for other papers by Fairfield T. Bain in
Current site
Google Scholar
PubMed
Close
 DVM, DACVIM, DACVP, DACVECC
,
Douglas T. ByarsHagyard Equine Medical Institute, 4250 Iron Works Pike, Lexington, KY 40511.

Search for other papers by Douglas T. Byars in
Current site
Google Scholar
PubMed
Close
 DVM, DACVIM, DACVECC
,
William V. BernardRood and Riddle Equine Hospital, 2150 Georgetown Rd, Lexington, KY 40511.

Search for other papers by William V. Bernard in
Current site
Google Scholar
PubMed
Close
 DVM, DACVIM
,
Martin O. FurrMarion DuPont Scott Equine Medical Center, 17690 Old Waterford Rd at Morven Park, PO Box 1938, Leesburg, VA 20177.

Search for other papers by Martin O. Furr in
Current site
Google Scholar
PubMed
Close
 DVM, PhD, DACVIM
,
Mary Rose ParadisDepartment of Clinical Sciences, School of Veterinary Medicine, Tufts University, North Grafton, MA 01536.

Search for other papers by Mary Rose Paradis in
Current site
Google Scholar
PubMed
Close
 DVM, MS, DACVIM
,
Jacquelin LawlerPeterson Smith Equine Hospital, 4747 Southwest 60th Ave, Ocala, FL 34474.

Search for other papers by Jacquelin Lawler in
Current site
Google Scholar
PubMed
Close
 DVM
,
Steeve GiguèreDepartment of Large Animal Clinical Sciences, College of Veterinary Medicine, University of Florida, Gainesville, FL 32610.

Search for other papers by Steeve Giguère in
Current site
Google Scholar
PubMed
Close
 DVM, PhD, DACVIM
, and
Bettina DunkelNew Bolton Center Campus, Department of Clinical Studies, School of Veterinary Medicine, University of Pennsylvania, Kennett Square, PA 19348.

Search for other papers by Bettina Dunkel in
Current site
Google Scholar
PubMed
Close
 DVM, DACVIM
View More View Less

Abstract

Objective—To create a mathematical model to assist in early prediction of the probability of discharge in hospitalized foals ≤ 7 days old.

Study Design—Prospective study.

Animals—1,073 foals.

Procedures—Medical records from 910 hospitalized foals ≤ 7 days old for which outcome was recorded as died or discharged alive were reviewed. Thirty-four variables including historical information, physical examination findings, and laboratory results were examined for association with survival. Variables associated with being discharged alive were entered into a multivariable logistic regression model. Accuracy of the model was validated prospectively on data from 163 foals.

Results—Factors in the final model included age group, ability to stand, presence of a suckle reflex, WBC count, serum creatinine concentration, and anion gap. Sensitivity and specificity of the model to predict live discharge were 92% and 74%, respectively, in the retrospective population and 90% and 46%, respectively, in the prospective population. Accuracy of an equine clinician's initial prediction of the foal being discharged alive was 83%, and accuracy of the model's prediction was 81%. Combining the clinician's prediction of probability of live discharge with that of the model significantly increased (median increase, 12%) the accuracy of the prediction for foals that were discharged and nonsignificantly decreased (median decrease, 9%) the accuracy of the predication for nonsurvivors.

Conclusions and Clinical Relevance—Combining the clinician's initial predication of the probability of a foal being discharged alive with that of the model appeared to provide a more precise early estimate of the probability of live discharge for hospitalized foals.

Abstract

Objective—To create a mathematical model to assist in early prediction of the probability of discharge in hospitalized foals ≤ 7 days old.

Study Design—Prospective study.

Animals—1,073 foals.

Procedures—Medical records from 910 hospitalized foals ≤ 7 days old for which outcome was recorded as died or discharged alive were reviewed. Thirty-four variables including historical information, physical examination findings, and laboratory results were examined for association with survival. Variables associated with being discharged alive were entered into a multivariable logistic regression model. Accuracy of the model was validated prospectively on data from 163 foals.

Results—Factors in the final model included age group, ability to stand, presence of a suckle reflex, WBC count, serum creatinine concentration, and anion gap. Sensitivity and specificity of the model to predict live discharge were 92% and 74%, respectively, in the retrospective population and 90% and 46%, respectively, in the prospective population. Accuracy of an equine clinician's initial prediction of the foal being discharged alive was 83%, and accuracy of the model's prediction was 81%. Combining the clinician's prediction of probability of live discharge with that of the model significantly increased (median increase, 12%) the accuracy of the prediction for foals that were discharged and nonsignificantly decreased (median decrease, 9%) the accuracy of the predication for nonsurvivors.

Conclusions and Clinical Relevance—Combining the clinician's initial predication of the probability of a foal being discharged alive with that of the model appeared to provide a more precise early estimate of the probability of live discharge for hospitalized foals.

The cost of hospitalization for neonatal foals is high because of the intensive treatment, cost of equipment necessary to monitor patients, and time commitment of hospital personnel. An estimate of the cost of daily care in 1998 at a university referral hospital ranged from $250 to > $1,000/d.1 The time required for treatment and monitoring of neonatal foals can range from a few hours up to 24 h/d. It is therefore imperative that accurate quantitative information regarding the probability for survival be provided to clients early in the course of hospitalization so they can make informed choices regarding case management options. Along with the quantitative probability of survival, the decision to treat may also be based on the estimated cost of treatment, value of the foal, effect of the condition on future performance, and personal priorities of the client.

Clinicians can make an initial estimate of the probability of survival in neonatal foals by assessment of history, clinical examination, and selected laboratory test results that are readily available early in the course of hospitalization. The use of a mathematical model to assist in establishing an early estimate of the probability of survival has the advantage of combining the clinician's estimate, based on prior experience, with quantitative information gained from observations of a large number of hospitalized neonates.

A mathematical model used to predict survival has features that are similar to any other diagnostic test. The accuracy of the result of a diagnostic test for the individual patient depends on the situation in which it is used.2 Therefore, to interpret a diagnostic test, the clinician must first decide how likely it is that the foal will survive on the basis of an initial evaluation. This estimate does not have to be precise but must distinguish foals with a low probability from those with a high probability of survival, and it must be expressed as a probability from 0 to 1. For example, given a large group of foals with historical and clinical examination findings similar to the patient being evaluated, what proportion of those foals would be expected to survive? The clinician's estimate of the probability of survival can then be combined with the results of the diagnostic test (ie, the mathematical model), by use of the sensitivity and specificity based on a single cut-point, to predict whether or not the foal will survive or to generate an adjusted quantitative probability of survival. Increased accuracy for the estimate of the probability of survival can be gained by the use of LRs.3 Likelihood ratios are a series of constants that can be combined with the clinician's estimate of the pretest odds of survival to provide a more precise posttest estimate of the probability of survival. Likelihood ratios have properties and provide information similar to sensitivity and specificity; however, LRs take into consideration the degree of abnormality, based on a range of probabilities for survival from 0 to 1, that can be generated by the model to further refine the estimate of survival for a specific foal.

Previous studies have identified risk factors for survival in neonatal foals and combined those factors into multivariable logistic regression models to predict a probability of survival. Those studies have been characterized by small sample sizes and have focused on specific subpopulations such as foals with septicemia,1 septic arthritis,4 and radiographic evidence of pulmonary disease,5 and foals admitted to ICUs.6,7 The results of those studies cannot be directly compared because the age of foals and selection criteria for inclusion varied considerably. To the authors' knowledge, there are no published studies that include a large, unselected population and incorporate the use of LRs to estimate the probability of survival in hospitalized foals ≤ 7 days of age. Although a more homogeneous sub group of foals may result in improved accuracy of the model, the advantage of using an unselected population of foals is that results of the model can be applied to a wider population of foals and the clinician does not have to decide whether the foal being evaluated is septicemic or meets the somewhat vague criteria for admission to an ICU.

The purpose of the study reported here was to develop a mathematical model that can be used by clinicians shortly after a foal ≤ 7 days of age is hospitalized to improve the estimate of the probability of survival.

Materials and Methods

A nonconcurrent prospective study design was used to develop the predictive model. Medical records of foals ≤ 7 days of age that were hospitalized for any reason at 2 private equine veterinary hospitals, HEMI or RREH in Lexington, Ky, or VTH-CVM-UT in Knoxville, Tenn, and had an outcome recorded as died or discharged alive from the hospital from 2000 to 2002 were eligible for inclusion in the study. Information on selected historical, demographic, initial physical examination, and laboratory findings that could be obtained on admission or within 2 hours after hospitalization were extracted from the medical records. Information for the mare included illness, premature labor, dystocia, induced parturition, and caesarian section. Data on the following variables were collected for the foal: hospital; age; location within hospital; treatment prior to hospitalization; duration of signs; provision of critical care; sucking reflex; recumbency; respiratory rate; rectal temperature; heart rate; RBC count; WBC count; total neutrophil, band, lymphocyte, and monocyte counts; neutrophil-to-lymphocyte ratio (calculated); chloride, potassium, and sodium concentrations; anion gap (calculated); immunoglobulin concentration; fibrinogen concentration; creatinine concentration, glucose concentration; total protein concentration; CO2 concentration; and outcome. Survival was defined as a foal that was discharged alive from the hospital. The medical records did not include information on the reason for euthanasia; therefore, records of foals that were euthanized were excluded from the analyses to eliminate the possibility of misclassification (eg, those foals that may have survived but were euthanized for economic or other reasons).

To construct the original model, variables were examined via univariate analysis for an association with outcome (died or discharged alive). Categoric data were compared by use of a c2 analysis or Fisher exact test,a depending on whether an expected cell value was < 5. Continuous variables were assessed by use of ANOVAb or a nonparametric equivalent,c depending on the distribution of the data. Variables associated with survival in the univariate analyses were entered into a logistic regression modeld in a forward stepwise manner. Order of entry into the model was based on a lower P value from the univariate analysis and the greater plausibility of an association between the variable and survival. When a continuous variable had a nonlinear association with survival, foals were ranked from lowest to highest value for the variable and allocated into 10 ordinal categories or groups of approximately equal size. The range of values for the continuous variable associated with each group of foals was recorded, and the proportion of foals that survived within each group was calculated. Adjacent categories with similar survival rates were collapsed when a natural break in survival rate was observed, and a sequential number was assigned beginning with 1 for the category with the lowest survival rate and increasing by 1 unit on the basis of increasing survival rate. Dichotomous characteristics (eg, presence or absence of a suckle reflex or ability to stand) were given values of 0 for absence or 1 for the presence of the factor. Because of multiple degrees of freedom, the P value associated with the Wald statistic was not used as the sole criteria for retention of a variable in the final model but was used in conjunction with the existence of a biologically plausible explanation for the association between the variable in question and outcome and the effect on model-fit statistics.8 Six variables were retained as main effects in the model. All 2-way interactions among the 6 main effects were added to the model and retained or removed by use of a backward stepwise procedure. Order of elimination was based on the P value associated with the interaction term after adjustment for other variables in the model. Interactions with the highest P value were eliminated first. The fit of the model to the data was assessed by use of Hosmer-Lemeshow goodness of fit and −2 log likelihood fit statistics. Prematurity, FPT, and illness in the mare were considered potential confounders and forced into the final model to evaluate significance and the effect on model-fit statistics.

A receiver operator characteristic curvee was generated, and the optimal cut-point to predict survival (sensitivity) and nonsurvival (specificity) was chosen on the basis of the estimate of test accuracy at each cut-point when calculated from the receiver operator characteristic output data set.3,d Briefly, sensitivity was defined as the number of foals that the model accurately predicted to survive (true-positive results) divided by the number of foals that survived (true-positive plus falsenegative results). Specificity was defined as the number of foals accurately predicted by the model to die (true-negative results) divided by the number of foals that died (true-negative plus false-positive results). Positive predictive value was the number of foals that were accurately predicted to survive (true-positive results) divided by the number of foals that the model predicted to survive (true-positive plus false-positive results), and the negative predictive value was the number of foals that the model accurately predicted to die (true-negative results) divided by the number of foals that the model predicted to die (true-negative plus false-negative results).

Accuracy of the model was defined as the number of foals accurately classified by the model (true-positive plus truenegative results) divided by the number of foals to which the model was applied.

Likelihood ratios were estimated by first ranking foals from the lowest to the highest probability of survival on the basis of output from the model. Foals were then grouped by placing them in ordinal categories consisting of each fifth percentile. The odds of survival associated with each of the ordinal categories were used to calculate LRs. Categories with similar LRs were collapsed into a single category. The formula used to calculate LRs was as follows: (No. of survivors within a given ordinal category/No. of foals that survived)/(No. of nonsurvivors in the same ordinal category/No. of nonsurvivors).3

A prospective study was designed, and data were collected during 2004 to validate the predictive model. Data collection forms were provided to equine practitioners from 1 private veterinary practice and 4 veterinary teaching hospitals: Peterson Smith Equine Hospital in Ocala, Fla; Texas A&M University in College Station, Tex; Tufts University in Boston, Mass; University of Pennsylvania in Kennett Square, Pa; and University of Florida in Gainesville, Fla. Participating veterinarians recorded outcome and the 6 characteristics of the hospitalized foals that were associated with survival in the original multivariable model. These included age group, presence of a suckle reflex, ability to stand, WBC count, serum creatinine concentration, and anion gap. Participating veterinarians were also asked to record their initial estimate of the probability of survival and expected outcome (eg, whether they thought the foal would survive or die) and to record the final outcome as follows: died, euthanized because of impending death (in the opinion of the clinician), euthanized for reasons other than impending death, or discharged alive. Data from foals that were euthanized for reasons other than impending death were removed from the analyses. Data from the remaining foals were used to validate the original predictive model.

Sensitivity, specificity, positive and negative predictive values, and accuracy of the model to predict survival in this population of patients were determined. The percentage of agreement beyond chance between the clinician and the model was expressed as the kappa coefficient (k), and the McNemar test was used to evaluate the significance of agreement beyond that expected by chance alone. The probability of survival as estimated by the clinician was compared with the probability of survival based on output from the model by use of the clinician's estimate as the pretest probability of survival and LRs generated from the model to estimate the posttest probability of survival. The difference in the probability of survival as estimated by the clinician and that generated by use of the clinician's estimate as the pretest probability of survival, combined with the LRs generated from the model, was tested for significance by use of a paired signed rank test.f Median differences in probability estimates for foals that survived and nonsurvivors were reported.

Data from both studies were entered directly into a computerized database file.g After all data were entered, the database file was imported into a separate file for statistical analyses.h For final analyses, P < 0.05 was considered significant.

Results

For the retrospective portion of the study, medical records were evaluated for 1,050 foals; records of 1,020 foals met inclusion criteria. One hundred ten of 1,020 foals were euthanized, so records of 910 foals were eligible for construction of the model. Variables included in the final model were age group, presence or absence of a suckle reflex, ability to stand, WBC count, serum creatinine concentration, and anion gap. Complete data on all 6 variables included in the final model were available for 577 of 910 (63%) foals, of which 479 survived and 98 died. Survival rate for this population was 83%.

Of the 577 foals with complete data, 371 (64%) were treated at HEMI, 187 (32%) were treated at RREH, and 19 (3%) were treated at VTH-CVM-UT. Survival rates among the 3 hospitals were not significantly (P = 0.15) different and were 82% for HEMI, 86% for RREH, and 68% for VTH-CVM-UT. Median ages of foals were also not significantly (P = 0.38) different among hospitals. Results of the univariate analyses to identify historical, physical examination, and laboratory findings associated with survival in foals were tabulated (Tables 1 and 2). Surviving foals were more likely than nonsurvivors to have a higher temperature, be older, have had previous treatment for their illness, have had a longer duration of clinical signs prior to hospitalization, and have a suckle reflex. Surviving foals were less likely than nonsurvivors to be assigned to receive critical care, be premature, be associated with dystocia or caesarian section, or be recumbent. Surviving foals also had significantly higher mean values than nonsurvivors for WBC count, total neutrophil count, neutrophil-to-lymphocyte ratio, blood glucose concentration, and proportion with IgG concentration ≥ 800 mg/100 mL, whereas mean values for anion gap and serum creatinine concentration were significantly lower. There was a significant association among foals < 48 hours old that lacked a suckle reflex with FPT (odds ratio, 2.8 [95% confidence interval, 1.5 to 5.2]).

Table 1—

Historical and clinical findings in mares and foals (No. [%]) evaluated for an association with survival of hospitalized foals ≤7 days of age (univariate analysis).

VariableSurvived (n = 744)*Died (166)**OR95% CI
Mare
   Illness
      No597 (82)132 (18)1.020.56, 1,88
      Yes62 (82)14 (18)RCRC
   Premature labor
      No612 (85)104 (15)3.182.11, 4.80
      Yes87 (65)47 (35)RCRC
   Dystocia
      No618 (83)   
      Yes88 (73)23 (27)RCRC
   Induced parturition
      No666 (82)145 (18)0.65 
      Yes7 (88)1 (13)RCRC
   Caesarian
      No661 (83)136 (17)6.082.35, 15.68
      Yes8 (44)10 (56)RCRC
Foal
   Age
      ≥1 day440 (90)49 (10)3.472.37, 5.08
      <1 day304 (72)117 (28)RCRC
   Location in hospital
      ICU345 (75)117 (25)1.0RC
      Barn176 (87)26 (13)2.31.41, 3.75
      Isolation161 (96)7 (4)7.803.55, 20.24
   Previous treatment
      Yes356 (88)48 (12)2.381.60, 3.53
      No306 (76)98 (24)RCRC
   Critical care    
      No174 (93)14 (7)3.051.70, 5.49
      Yes399 (80)98 (20)RCRC
   Suckling reflex
      Present596 (92)50 (7)12.618.19, 19.44
      Absent87 (49)92 (51)RCRC
   Recumbent    
      No545 (96)24 (4)16.6410.20, 27.34
      Yes176 (58)129 (42)RCRC
   Respiratory rate
      ≤20 breaths/min699 (84)134 (16)4.061.86, 8.81
      <20 breaths/min18 (56)14 (44)RCRC
   Rectal temperature (°F)
      ≤96.242 (45)52 (55)1.0RC
      >96.2, ≤99.3104 (67)52 (33)2.481.42, 4.34
      >99.3602 (91)62 (9)12.027.21, 20.08
   Heart rate    
      ≥76 beats/min661 (84)127 (16)2.601.45, 4.64
      <76 beats/min44 (67)22 (33)RCRC

Denominators may vary because of missing data.

Euthanized foals were excluded from the analyses.

OR = Odds ratio. CI = Unadjusted 95% confidence interval of the OR. RC = Referent category.

To convert temperatures to degrees Celsius, subtract 32 and multiply by 5/9.

Table 2—

Hematologic variables with a significant (P < 0.05 [univariate analysis]) nonlinear association with survival in hospitalized foals < 7 days of age (No. [%]).

VariableSurvived (n = 744)*Died (166)*OR95% CI
Total WBC (× 103 cells/μ L)
      <2,00035 (43)47 (57)1RC
      ≥2,000, <4,370129 (73)48 (27)3.612.01, 6.49
      ≥4,370555 (90)63 (10)11.836.90, 20.34
Total neutrophils (× 103 cells/μ L)
      ≥2,530427 (91)40 (9)7.534.78, 11.90
      <2,530112 (59)79 (41)RCRC
Total lymphocytes (× 103 cells/μ L)
      ≥528 and < 1,988399 (86)66 (14)2.321.51, 3.57
      ≥1,988 or <528138 (72)53 (28)RCRC
Neutrophil-to-lymphocyte ratio
      <0.6923 (38)38 (62)1RC
      ≥0.69 and <1.3839 (61)25 (39)2.581.18, 5.67
      ≥1.38 and <2.0352 (75)17 (25)5.052.23, 11.57
      >2.03423 (92)39 (8)17.929.31, 34.71
Sodium (mEq/L)
      <143608 (85)111 (15)4.022.50, 6.47
      ≥14349 (58)36 (42)RCRC
Potassium (mEq/L)
      <5616 (85)106 (15)5.813.48, 9.70
      ≥540 (50)40 (50)RCRC
Total CO2 (mg/dL)
      ≥19576 (85)104 (15)2.711.78, 4.14
      <1998 (67)48 (33)RCRC
Anion gap (mmol/L)
      ≥23.632 (45)39 (55)1RC
      ≥17 and <23.6122 (75)41 (25)3.631.94, 6.81
      ≥15.7 and <17128 (84)25 (16)6.243.16, 12.40
      <15.7357 (90)38 (10)11.456.20, 21.22
Creatinine (mg/dL)
      ≥4118 (63)68 (37)1RC
      ≥1.9 and <4163 (83)33 (17)2.851.72, 4.73
      <1.9276 (95)16 (5)9.945.36, 18.67
Glucose (mg/dL)
      <5648 (44)60 (56)1RC
      ≥56 and <10485 (79)22 (21)4.832.54, 9.26
      ≥104328 (92)27 (8)15.198.51, 27.26

See Table 1 for key.

Variables that were significant in the univariate analysis were entered into a logistic regression model in a forward stepwise manner. Variables retained in the final model included age group, WBC count, ability to stand, presence of a suckle reflex, anion gap, and serum creatinine concentration (Table 3). The final model had an R2 value of 0.52, a P value of 0.45 associated with the Hosmer-Lemeshow goodness-of-fit test, and a −2 log likelihood fit test value of 312, indicating a good fit of the model to the data. The sensitivity and specificity to predict the survival or nonsurvival at different cut-points (probabilities) that were generated by the model were determined (Figure 1). By use of a cutpoint of 0.64877, the sensitivity of the model was 92% and specificity was 74%. Positive and negative predictive values for this population of foals were 95% and 65%, respectively (Table 4). Accuracy of the model ranged from 33% to 89%, and the choice of a cut-point to distinguish foals that survive from nonsurvivors was made to maximize accuracy.

Table 3—

Results of a multivariable analysis to identify the optimal combination of characteristics for predicting survival in hospitalized foals ≤7 days of age (n = 577).

VariableCoefficientSEP valueOR for survival95% CI
Intercept−3.41800.7116<0.001NANA
Age group−1.77301.05610.0930.1700.021, 1.346
R2.07160.3855<0.0017.9373.728, 16.899
S1.05620.3167<0.0012.8751.546, 5.350
Cr0.49560.21800.0231.6411.071, 2.517
AG0.2552    
WBC0.72950.25520.0040.0741.258, 3.420
Age group × WBC0.7653    

R = Recumbent. S = Suckle reflex. Cr = Serum creatinine concentration. AG = Anion gap.

Probability of survival = eA/(1 + eA), where A = −3.4180 – AGE GROUP × 1.7730 + R × 2.0716 + S × 1.0562 + Cr × 0.4956 + AG × 0.2552 + WBC × 0.7295 + AGE GROUP × WBC × 0.7653. Values for variables were assigned as follows: if WBC < 2,000, value = 1; if WBC ≥ 2,000 < 4,370, value = 2; if WBC ≥ 4,370, value = 3; if AGE < 1 day, value = 0; if AGE ≥ 1 day, value = 1; if S absent, value = 0; if S present, value = 1; if R, value = 0; if not R, value = 1; if AG > 23.6, value = 1; if AG ≥ 17 < 23.6, value = 2; if AG ≥ 15.7 < 17, value = 3; if AG < 15.7, value = 4; if Cr ≥ 4, value = 1; if Cr ≥ 1.9 and < 4, value = 2; if Cr < 1.9, value = 3; if probability ≥ 0.64877, then predict survival; and if probability < 0.64877, then predict nonsurvival. Goodness of fit tests: R2 = 0.52; Hosmer and Lemeshow, P = 0.45; −2 log likelihood = 312. OR represents the increased odds of survival associated with each incremental increase in category. NA = Not Applicable. See Table 1 for remainder of key.

Table 4—

Accuracy (% [95% CI]) of a multivariable model to predict survival in hospitalized foals ≤ 7 days of age in a retrospective sample of foals (n = 577) used to develop the model and a prospective sample of foals (163) from a distinct population used to validate the model.

Variable  
Sensitivity92 (89–94)90 (85–95)
Specificity74 (66–83)46 (27–65)
Positive predictive value95 (93–97)90 (85–95)
Negative predictive value65 (56–74)46 (27–65)
Test accuracy89 (86–91)83 (77–87)
Proportion of foals surviving8384
Figure 1—
Figure 1—

Receiver operator characteristic curve illustrating the sensitivity and specificity at various cut-points for a multivariable logistic regression model to predict survival in hospitalized foals (n = 577) ≤ 7 days of age.

Citation: Journal of the American Veterinary Medical Association 228, 11; 10.2460/javma.228.11.1748

Validation of the model was based on data for 163 foals from 5 hospitals. When applied to the new populations, the model was able to accurately predict survival in 90% of foals (sensitivity) and predict nonsurvival in 46% of foals (specificity). The proportion of foals that survived was 137 of 163 (84%), and the positive and negative predictive values were 90% and 46%, respectively. The proportions of foals that survived were similar among the 5 hospitals (P = 0.7). Data on the clinician's prediction as to whether the foal would survive were available for 123 of 163 (75%) foals and were compared with the prediction based on the model. The accuracies of the clinician and model were similar (83% and 81%, respectively [P = 0.8]), but the predictions were not well correlated (k = 0.29; P = 0.07).

When the equine clinician's initial estimate of survival was combined with output from the model, estimates of the probability of survival in foals that were discharged alive increased by a median probability of 12% (P < 0.001) and estimates of survival for those foals that died had decreased by a median probability of 9%, which was not significant (P = 0.7). Likelihood ratios were calculated for various ranges of probabilities generated by the retrospective model on the population of foals from HEMI, RREH, and VTH-CVM-UT (Table 5).

Table 5—

Likelihood ratios (LR) based on selected ranges of probabilities generated from a multivariable logistic regression model to predict survival in hospitalized foals ≤ 7 days of age.

ValueSurvivedDiedLR
< 0.236483 (0.63)26 (26.53)0.0236
≥ 0.23648, < 0.3906512 (2.51)17 (17.35)0.1444
≥ 0.39065, < 0.6862232 (6.68)30 (30.61)0.2182
≥ 0.68622, < 0.7821320 (4.18)6 (6.12)0.6820
≥ 0.78213, < 0.8744546 (9.60)9 (9.18)1.0457
≥ 0.87445, < 0.9339631 (6.47)3 (3.06)2.1141
≥ 0.93396, < 0.97479106 (22.13)5 (5.10)4.3374
≥ 0.97479229 (47.81)2 (2.04)23.4259
Total47998

Discussion

The accuracy of the model was determined prospectively by predicting survival in foals subsequently admitted to 5 hospitals; however, the utility of the model was based on the ability to combine information from the clinician's initial assessment with data from observations of 577 foals to improve the accuracy of prognostic information given to the client early in the course of hospitalization. When both sources of information were combined, there was a modest but significant increase in the accuracy of the estimate of probability of survival among foals that survived and a nonsignificant decrease in the accuracy of the estimate of probability of nonsurvival among foals that died.

Survival rates of 83% and 84% were observed in the sample of foals used to develop the predictive model and the sample used to validate the model, respectively. Median age of foals in the retrospective and prospective populations were also similar. This suggested that the sample of foals used to construct the model was representative of the general population of hospitalized foals from veterinary teaching hospitals and private equine hospitals. In addition, there were no significant differences in the proportions of foals that survived among the 5 hospitals that enrolled foals in the prospective study. The survival rate for study foals was higher than that found in other published studies.6,7 In the development of 2 previous models designed to predict survival in foals admitted to neonatal ICUs, the reported survival rates were 69% and 66% for the sample of foals used to create the models and 76% and 50% for the sample of foals used to validate the models.6,7 The survival rate in a sample of foals used to develop a model to predict survival among septicemic foals was 45%.1 Given the worst-case scenario—that all foals that were euthanized would have died in both the retrospective and prospective populations—survival rates would have been 73% and 74%, respectively. These values are still higher than those reported for all but one of the groups mentioned in prior studies. Foals admitted to ICUs and septicemic foals represent subpopulations of hospitalized neonates selected because of severe disease. The differences in survival rates among these populations and the foals in this study can be explained by the fact that foals in this study were unselected and therefore represented a more complete spectrum from mild to severe illness.

Survival models in general are better at predicting outcome when validated prospectively with foals at the same hospital from which the model was created.6 In a previous study,6 validation of a foal survival model on a new group of foals from the same teaching hospital ICU resulted in a sensitivity of 83% and specificity of 87%. However, when the model was applied to a group of foals from a second teaching hospital ICU, the sensitivity remained at 83% but the specificity was reduced to 44%.6 In another study,7 validation of the model was performed by use of a new group of foals from the same university ICU; the sensitivity decreased from 92% to 83%, and the specificity decreased from 100% to 40%. One explanation for the increased variability in specificity (ability of the model to accurately classify nonsurvivors) when the model was validated with a different population of foals may be the inclusion of foals that were euthanized because of poor prognosis, leading to the possibility of misclassification with regard to survival. In addition, a larger proportion of foals usually survive, resulting in a smaller sample size of nonsurvivors from which to estimate the model specificity. We were able to validate the model with a group of foals that were admitted to 4 veterinary teaching hospitals and 1 private veterinary hospital distinct from the 3 hospitals used to create the model; however, only 26 of 163 (16%) of these foals were either euthanized because of impending death or died and therefore could be used to estimate specificity. The positive predictive value of the model in the retrospective population (95%) was similar to that in the prospective population (90%); however, the negative predictive value decreased from 65% to 46%. A probability of survival that is ≥ 90% represents valuable information that can be communicated to a client struggling with the decision of whether to embark on an intensive and costly treatment program. Just as important is the substantial (46%) probability of nonsurvival associated with a negative test result. The results of the prospective study suggested that the model was robust, and the results are likely to be applicable to foals admitted to private equine hospitals as well as veterinary teaching hospitals.

When adjusted for other factors in the model, foals that survived were more likely to have a suckle reflex. Absence of a suckle reflex may be caused by systemic illness or injury and result in inadequate intake of colostrum. If the illness or injury occurs in utero or within a few hours after birth, FPT may occur and decrease the foal's resistance to infection. 9 Many of the foals in the present study were young; 383 of 577 (66%) were < 48 hours old, and there was a significant association with FPT among foals < 48 hours of age that lacked a suckle reflex. However, when forced into the model, FPT was not significant and did not improve the fit of the model to the data. A study10 of mortality rate in an intensively managed mare herd concluded that a main factor in foal survival was FPT. Contrary to findings in the present study, findings of 2 of 3 prior studies, 1 on survival of septic foals1 and the other on foals admitted to an ICU,6 found no association between suckle reflex and survival in univariate analyses and this variable was not included in the final models. The ages of foals in the 2 studies were < 4 weeks and < 14 days. If the suckle reflex becomes suppressed or absent after 2 days of age, foals are less likely to develop FPT, and lack of this reflex after 2 days is more likely to represent severe neurologic disease or circulatory shock. The third study7 of foals < 10 days of age admitted to an ICU did not include suckle reflex in the data analysis.

The factor most strongly associated with survival in study foals was the ability to stand on admission. This association has been reported in previous studies, suggesting that events leading to recumbency are related to survival. Foals with neonatal maladjustment syndrome that had a normal gestational length and delivery and were able to stand and walk had a much better (up to 80%) prognosis.11 In septic foals < 4 weeks of age, survival was greater in foals standing on admission to the hospital than those unable to stand.1 Prolonged recumbency may be related to the severity of illness and shock in which survival is likely to be lower. Dependent atelectasis may occur after prolonged recumbency because of fluid that infiltrates the pulmonary interstitium.12 A recumbent foal that is ≤ 24 hours old and has been unable to nurse is subject to FPT and nutritional deprivation, which can further weaken the foal and suppress the immune system.

Median serum creatinine concentration in study foals that survived was 2.0 mg/dL (interquartile range, 1.4 to 3.6 mg/dL) and in nonsurvivors was 4.2 mg/dL (interquartile range, 2.7 to 6.6 mg/dL). In foals < 4 weeks of age with radiographic evidence of pulmonary disease,5 foals that died were 4.9 times as likely to have serum creatinine concentrations > 1.7 mg/dL, and surviving foals < 14 days of age admitted to an ICU6 had a lower mean creatinine concentration (2.79 mg/dL) than those that died (4.60 mg/dL); however, in the latter study, serum creatinine concentration did not retain significance and was subsequently removed from the final multivariable model. In separate studies of septic foals1 and those admitted to an ICU,7 creatinine concentration was measured but did not reveal significance in the univariate analysis. In the latter 3 studies, foals were selected for severe disease (eg, admission to an ICU or sepsis) and were older than foals in the present study. Excessive serum creatinine concentration in a neonatal foal may be caused by muscle atrophy; renal disease; dehydration; or problems with the mare, including high creatinine concentration in mare serum or inadequate placental function.13 Less common conditions associated with high creatinine concentration include caesarian section, placentitis, induced parturtion, and premature placental separation. Median age for foals in the present retrospective case series was 1 day, at which time complications involving parturition are more likely to be observed. The inconsistencies in the association between serum creatinine concentration and survival among previous studies and the present study may be related to differences in the age of foals and the differing clinical criteria for inclusion in each of the studies. High serum creatinine concentration appears to be a better marker to separate foals ≤ 7 days of age with severe metabolic disease, and thus less likely to survive, from those with mild or moderate disease in an unselected population.

The proportion of foals that survived after hospitalization was directly related to increasing WBC, as designated by ordinal categories from 1 to 3. Univariate analyses in 3 prior studies1,6,7 revealed that WBC counts were consistently higher among survivors than nonsurvivors. In the present study, surviving foals had a median WBC count of 7.4 × 103 cells/mL and nonsurvivors had 3.3 × 103 cells/mL. Despite substantial differences in WBC counts reported in the first 2 of the previous studies, a significant difference was only attained in the second study.6 Lack of significance in the first study1 may have been related to the small sample size and variability associated with WBC counts. In the third study,7 mean WBC counts were high in survivors and nonsurvivors. In that study, survival rate was 66% (n = 56), versus 83% (577) in the present study. White blood cell counts among survivors and nonsurvivors in the third study were higher than those of survivors in the 2 prior studies and the present study. This suggests that the health-related events of foals admitted to the ICU in the third study differed substantially.

Two previous studies included anion gap as a factor in multivariable models to predict survival in foals with radiographic evidence of pulmonary disease5 and foals < 10 days of age admitted to an ICU.7 Of 2 previous studies of adult horses with colic, 1 study14 did not include anion gap in the final multivariate analysis; however, in the second study,15 increasing values of anion gap correlated well with a decrease in survival rate. High anion gap is associated with shock, diabetic ketoacidosis, and renal failure in horses16 and is an indicator of metabolic acidosis in foals.17 Metabolic acidosis is a common condition in ill neonatal foals18 and is most commonly seen when tissue perfusion and oxygen delivery are decreased. Causes of decreased tissue oxygen delivery include low cardiac output secondary to sepsis and asphyxia, pulmonary disease with severe hypoxemia, and reduced oxygen carrying capacity of the blood (severe anemia).18

The logistic regression model expresses the probability of survival on a continuous scale from 0% to 100%. Use of a single cut-point to describe the sensitivity and specificity of the model to predict survival in a hospitalized neonate can result in loss of information and distortion.19 Use of a single cut-point carries with it the assumption that all foals given a probability of > 0.64877 on the basis of output from the model have the same probability of survival, and similarly, all foals with a probability of ≤ 0.64877 are given the same, albeit different, probability of survival. Thus, if only a single cut-point of > 0.64877 was assigned to distinguish survivors and nonsurvivors, a foal with a given probability of survival of 0.99 (99%) on the basis of output from the model would be given the same probability of survival as all other foals with values ranging from 0.65 to 1. The LR is a ratio of the proportion of foals that survived, given that the output from the model lies within a narrow range of probabilities, compared with the proportion of nonsurvivors given the same narrow range of values. By ranking foals from the retrospective portion of the study in order of probability of survival and dividing them into subgroups on the basis of a narrow range of probabilities, more information contained in the data can be utilized. Clinicians can combine their estimate of the probability of survival with observations of foals in the present study, not on the basis of a single cut-point, but rather on whether the model-generated probability for survival falls within a narrow range of values along the probability distribution.

Data for creation of the model were collected retrospectively and are not as likely to be as complete or accurate as if collected for the purpose of the study. Specific information on the reason for euthanasia was not in the medical records extracted retrospectively; therefore, these foals were excluded from analyses. In addition, it was not possible to document with certainty that foals had not received oxygen supplementation or fluid therapy prior to obtaining samples for laboratory analysis. The population of foals used to create the model could have been biased toward those with clinical signs that suggested that certain tests were performed. However, when the median age and survival rate of foals that were included in the retrospective portion of the study were compared with the sample of foals that were not euthanized but were excluded from the study because of lack of complete data, they were not significantly different. Foals that were euthanized in the prospective study were included in the analysis only if they were euthanized because of impending death. The model was accurate when validated on an unselected population of foals admitted to equine hospitals distinct from those used to generate the model. These observations suggest that selection bias did not play a role in the results and indicate that the study patients are likely to be representative of all hospitalized foals. Because of the low negative predictive value of the model when applied to the prospective population of foals, it alone should not be used as the sole criterion on which to base a decision for euthanasia.

However, knowledge that a foal has a 46% probability of nonsurvival should be included in the decisionmaking process. The possibility of confounding exists with all epidemiologic studies. This issue was discussed in a prior report,6 and illness in the mare, serum IgG concentration, and prematurity were mentioned as possible confounding factors. In the study reported here, all were forced into the final model, none were significant, and all reduced the fit of the model to the data.

The model has been incorporated into a spreadsheeti that is available upon request from the corresponding author. Practitioners may use it to combine their clinical impression of survival with the quantitative experience based on observations of a large sample of foals, giving a more accurate prediction of probability of survival during the first few hours of hospitalization. Field use of the model is problematic; if test results are not immediately available, a delay in the decision to refer to a hospital may affect prognosis, and clinical and laboratory characteristics measured at an earlier stage of disease may affect the accuracy of the model. Use of the model in an equine hospital setting can assist the client in making an informed decision regarding patient management and result in more efficient use of personnel and medical and surgical resources within the hospital.

ABBREVIATIONS

LR

Likelihood ratio

ICU

Intensive care unit

HEMI

Hagyard Equine Medical Institute

RREH

Rood and Riddle Equine Hospital

VTH-CVM-UT

Veterinary Teaching Hospital, College of Veterinary Medicine, University of Tennessee

FPT

Failure of passive transfer

a.

PROC FREQ, SAS 9.1 for Windows, SAS Institute Inc, Cary, NC.

b.

PROC TTEST, SAS 9.1 for Windows, SAS Institute Inc, Cary, NC.

c.

PROC NPAR1WAY, SAS 9.1 for Windows, SAS Institute Inc, Cary, NC.

d.

PROC LOGISTIC, SAS 9.1 for Windows, SAS Institute Inc, Cary, NC.

e.

PROC GPLOT, SAS 9.1 for Windows, SAS Institute Inc, Cary, NC.

f.

PROC UNIVARIATE, SAS 9.1 for Windows, SAS Institute Inc, Cary, NC.

g.

Access, Microsoft Corp, Redmond, Wash.

h.

SAS for Windows, version 9.1, SAS Institute Inc, Cary, NC.

i.

Excel, Microsoft Corp, Mountain View, Calif.

References

  • 1

    Gayle JM, Cohen ND, Chaffin MK. Factors associated with survival in septicemic foals: 65 cases (1988–1995). J Vet Intern Med 1998;12:140146.

    • Search Google Scholar
    • Export Citation
  • 2

    Romatowski J. Problems in interpretation of clinical laboratory test results. J Am Vet Med Assoc 1994;205:11861188.

  • 3

    Fletcher RH, Fletcher SW, Wagner EH. Clinical epidemiology: the essentials. 3rd ed. Baltimore: The Williams & Wilkins Co, 1996;4374.

  • 4

    Steel CM, Hunt AR, Adams PL, et al. Factors associated with prognosis for survival and athletic use in foals with septic arthritis: 93 cases (1987–1994). J Am Vet Med Assoc 1999;215:973977.

    • Search Google Scholar
    • Export Citation
  • 5

    Bedenice D, Heuwieser W, Solano M, et al. Risk factors and prognostic variables for survival of foals with radiographic evidence of pulmonary disease. J Vet Intern Med 2003;17:868875.

    • Search Google Scholar
    • Export Citation
  • 6

    Furr M, Tinker MK, Edens L. Prognosis for neonatal foals in an intensive care unit. J Vet Intern Med 1997;11:183188.

  • 7

    Hoffman AM, Staempfli HR, Willan A. Prognostic variables for survival of neonatal foals under intensive care. J Vet Intern Med 1992;6:8995.

    • Search Google Scholar
    • Export Citation
  • 8

    Hosmer DW, Lemeshow S. Applied logistic regression. New York: John Wiley & Sons, 1989;2537.

  • 9

    Clement SF. Convulsive and allied syndromes of the neonatal foal. Vet Clin North Am Equine Pract 1987;3:333344.

  • 10

    Haas SD, Bristol F, Card CE. Risk factors associated with the incidence of foal mortality in an extensively managed mare herd. Can Vet J 1996;37:9195.

    • Search Google Scholar
    • Export Citation
  • 11

    Clement SF. Behavioral alterations and neonatal maladjustment syndrome in the foal, in Proceedings. 31st Annu Meet Am Assoc Equine Pract 1985;145148.

    • Search Google Scholar
    • Export Citation
  • 12

    Lester GD, Lester NV. Abdominal and thoracic radiography in the neonate. Vet Clin North Am Equine Pract 2001;17:1946.

  • 13

    Brewer BD. Renal disease. In: Korterba AM, Drummond WH, Kosch PC, eds. Equine clinical neonatalogy. Philadelphia: Lea & Febiger, 1990;446461.

    • Search Google Scholar
    • Export Citation
  • 14

    Reeves MJ, Curtis CR, Salman MD, et al. Prognosis in equine colic patients using multivariable analysis. Can J Vet Res 1989;53:8794.

  • 15

    Bristol DG. The anion gap as a prognostic indicator in horses with abdominal pain. J Am Vet Med Assoc 1982;181:6365.

  • 16

    Gossett KA, French DD. Effect of age on anion gap in clinically normal Quarter Horses. Am J Vet Res 1983;44:17441745.

  • 17

    Meyer DJ, Harvey JW. Veterinary laboratory medicine: interpretation & diagnosis. 2nd ed. Philadelphia: WB Saunders Co, 1998;237247.

  • 18

    Spurlock SL, Furr M. Fluid therapy. In: Korterba AM, Drummond WH, Kosch PC, eds. Equine clinical neonatalogy. Philadelphia: Lea & Febiger, 1990;671700.

    • Search Google Scholar
    • Export Citation
  • 19

    Brown MD, Reeves MJ. Evidence-based emergency medicine/skills for evidence-based emergency care. Interval likelihood ratios: another advantage for the evidence-based diagnostician. Ann Emerg Med 2003;42:292297.

    • Search Google Scholar
    • Export Citation

Contributor Notes

Dr. Buchanan's present address is Brazos Valley Equine Hospital, 6999 Hwy 6, Navasota, TX 77868.

Dr. Byars' present address is 2183 Ironworks Pike, Georgetown, KY 40324.

Dr. Bernard's present address is Equine Internal Medicine, 4901 Mt Horeb, Lexington, KY 40511.

Dr. Lawler's present address is Department of Clinical Sciences, College of Veterinary Medicine and Biomedical Sciences, Colorado State University, Fort Collins, CO 80523.

Supported by the Department of Large Animal Clinical Sciences and Center of Excellence, University of Tennessee.

The authors thank Dr. Jennifer Collins for assistance with data collection and Dr. Arnold Saxton for statistical consultation.

Address correspondence to Dr. Rohrbach.
  • View in gallery
    Figure 1—

    Receiver operator characteristic curve illustrating the sensitivity and specificity at various cut-points for a multivariable logistic regression model to predict survival in hospitalized foals (n = 577) ≤ 7 days of age.

  • 1

    Gayle JM, Cohen ND, Chaffin MK. Factors associated with survival in septicemic foals: 65 cases (1988–1995). J Vet Intern Med 1998;12:140146.

    • Search Google Scholar
    • Export Citation
  • 2

    Romatowski J. Problems in interpretation of clinical laboratory test results. J Am Vet Med Assoc 1994;205:11861188.

  • 3

    Fletcher RH, Fletcher SW, Wagner EH. Clinical epidemiology: the essentials. 3rd ed. Baltimore: The Williams & Wilkins Co, 1996;4374.

  • 4

    Steel CM, Hunt AR, Adams PL, et al. Factors associated with prognosis for survival and athletic use in foals with septic arthritis: 93 cases (1987–1994). J Am Vet Med Assoc 1999;215:973977.

    • Search Google Scholar
    • Export Citation
  • 5

    Bedenice D, Heuwieser W, Solano M, et al. Risk factors and prognostic variables for survival of foals with radiographic evidence of pulmonary disease. J Vet Intern Med 2003;17:868875.

    • Search Google Scholar
    • Export Citation
  • 6

    Furr M, Tinker MK, Edens L. Prognosis for neonatal foals in an intensive care unit. J Vet Intern Med 1997;11:183188.

  • 7

    Hoffman AM, Staempfli HR, Willan A. Prognostic variables for survival of neonatal foals under intensive care. J Vet Intern Med 1992;6:8995.

    • Search Google Scholar
    • Export Citation
  • 8

    Hosmer DW, Lemeshow S. Applied logistic regression. New York: John Wiley & Sons, 1989;2537.

  • 9

    Clement SF. Convulsive and allied syndromes of the neonatal foal. Vet Clin North Am Equine Pract 1987;3:333344.

  • 10

    Haas SD, Bristol F, Card CE. Risk factors associated with the incidence of foal mortality in an extensively managed mare herd. Can Vet J 1996;37:9195.

    • Search Google Scholar
    • Export Citation
  • 11

    Clement SF. Behavioral alterations and neonatal maladjustment syndrome in the foal, in Proceedings. 31st Annu Meet Am Assoc Equine Pract 1985;145148.

    • Search Google Scholar
    • Export Citation
  • 12

    Lester GD, Lester NV. Abdominal and thoracic radiography in the neonate. Vet Clin North Am Equine Pract 2001;17:1946.

  • 13

    Brewer BD. Renal disease. In: Korterba AM, Drummond WH, Kosch PC, eds. Equine clinical neonatalogy. Philadelphia: Lea & Febiger, 1990;446461.

    • Search Google Scholar
    • Export Citation
  • 14

    Reeves MJ, Curtis CR, Salman MD, et al. Prognosis in equine colic patients using multivariable analysis. Can J Vet Res 1989;53:8794.

  • 15

    Bristol DG. The anion gap as a prognostic indicator in horses with abdominal pain. J Am Vet Med Assoc 1982;181:6365.

  • 16

    Gossett KA, French DD. Effect of age on anion gap in clinically normal Quarter Horses. Am J Vet Res 1983;44:17441745.

  • 17

    Meyer DJ, Harvey JW. Veterinary laboratory medicine: interpretation & diagnosis. 2nd ed. Philadelphia: WB Saunders Co, 1998;237247.

  • 18

    Spurlock SL, Furr M. Fluid therapy. In: Korterba AM, Drummond WH, Kosch PC, eds. Equine clinical neonatalogy. Philadelphia: Lea & Febiger, 1990;671700.

    • Search Google Scholar
    • Export Citation
  • 19

    Brown MD, Reeves MJ. Evidence-based emergency medicine/skills for evidence-based emergency care. Interval likelihood ratios: another advantage for the evidence-based diagnostician. Ann Emerg Med 2003;42:292297.

    • Search Google Scholar
    • Export Citation

Advertisement