Materials and Methods

A nonconcurrent prospective study design was used to develop the predictive model. Medical records of foals ≤ 7 days of age that were hospitalized for any reason at 2 private equine veterinary hospitals, HEMI or RREH in Lexington, Ky, or VTH-CVM-UT in Knoxville, Tenn, and had an outcome recorded as died or discharged alive from the hospital from 2000 to 2002 were eligible for inclusion in the study. Information on selected historical, demographic, initial physical examination, and laboratory findings that could be obtained on admission or within 2 hours after hospitalization were extracted from the medical records. Information for the mare included illness, premature labor, dystocia, induced parturition, and caesarian section. Data on the following variables were collected for the foal: hospital; age; location within hospital; treatment prior to hospitalization; duration of signs; provision of critical care; sucking reflex; recumbency; respiratory rate; rectal temperature; heart rate; RBC count; WBC count; total neutrophil, band, lymphocyte, and monocyte counts; neutrophil-to-lymphocyte ratio (calculated); chloride, potassium, and sodium concentrations; anion gap (calculated); immunoglobulin concentration; fibrinogen concentration; creatinine concentration, glucose concentration; total protein concentration; CO₂ concentration; and outcome. Survival was defined as a foal that was discharged alive from the hospital. The medical records did not include information on the reason for euthanasia; therefore, records of foals that were euthanized were excluded from the analyses to eliminate the possibility of misclassification (eg, those foals that may have survived but were euthanized for economic or other reasons).

To construct the original model, variables were examined via univariate analysis for an association with outcome (died or discharged alive). Categoric data were compared by use of a c² analysis or Fisher exact test,^a depending on whether an expected cell value was < 5. Continuous variables were assessed by use of ANOVA^b or a nonparametric equivalent,^c depending on the distribution of the data. Variables associated with survival in the univariate analyses were entered into a logistic regression model^d in a forward stepwise manner. Order of entry into the model was based on a lower P value from the univariate analysis and the greater plausibility of an association between the variable and survival. When a continuous variable had a nonlinear association with survival, foals were ranked from lowest to highest value for the variable and allocated into 10 ordinal categories or groups of approximately equal size. The range of values for the continuous variable associated with each group of foals was recorded, and the proportion of foals that survived within each group was calculated. Adjacent categories with similar survival rates were collapsed when a natural break in survival rate was observed, and a sequential number was assigned beginning with 1 for the category with the lowest survival rate and increasing by 1 unit on the basis of increasing survival rate. Dichotomous characteristics (eg, presence or absence of a suckle reflex or ability to stand) were given values of 0 for absence or 1 for the presence of the factor. Because of multiple degrees of freedom, the P value associated with the Wald statistic was not used as the sole criteria for retention of a variable in the final model but was used in conjunction with the existence of a biologically plausible explanation for the association between the variable in question and outcome and the effect on model-fit statistics.⁸ Six variables were retained as main effects in the model. All 2-way interactions among the 6 main effects were added to the model and retained or removed by use of a backward stepwise procedure. Order of elimination was based on the P value associated with the interaction term after adjustment for other variables in the model. Interactions with the highest P value were eliminated first. The fit of the model to the data was assessed by use of Hosmer-Lemeshow goodness of fit and −2 log likelihood fit statistics. Prematurity, FPT, and illness in the mare were considered potential confounders and forced into the final model to evaluate significance and the effect on model-fit statistics.

A receiver operator characteristic curve^e was generated, and the optimal cut-point to predict survival (sensitivity) and nonsurvival (specificity) was chosen on the basis of the estimate of test accuracy at each cut-point when calculated from the receiver operator characteristic output data set.^3,d Briefly, sensitivity was defined as the number of foals that the model accurately predicted to survive (true-positive results) divided by the number of foals that survived (true-positive plus falsenegative results). Specificity was defined as the number of foals accurately predicted by the model to die (true-negative results) divided by the number of foals that died (true-negative plus false-positive results). Positive predictive value was the number of foals that were accurately predicted to survive (true-positive results) divided by the number of foals that the model predicted to survive (true-positive plus false-positive results), and the negative predictive value was the number of foals that the model accurately predicted to die (true-negative results) divided by the number of foals that the model predicted to die (true-negative plus false-negative results).

Accuracy of the model was defined as the number of foals accurately classified by the model (true-positive plus truenegative results) divided by the number of foals to which the model was applied.

Likelihood ratios were estimated by first ranking foals from the lowest to the highest probability of survival on the basis of output from the model. Foals were then grouped by placing them in ordinal categories consisting of each fifth percentile. The odds of survival associated with each of the ordinal categories were used to calculate LRs. Categories with similar LRs were collapsed into a single category. The formula used to calculate LRs was as follows: (No. of survivors within a given ordinal category/No. of foals that survived)/(No. of nonsurvivors in the same ordinal category/No. of nonsurvivors).³

A prospective study was designed, and data were collected during 2004 to validate the predictive model. Data collection forms were provided to equine practitioners from 1 private veterinary practice and 4 veterinary teaching hospitals: Peterson Smith Equine Hospital in Ocala, Fla; Texas A&M University in College Station, Tex; Tufts University in Boston, Mass; University of Pennsylvania in Kennett Square, Pa; and University of Florida in Gainesville, Fla. Participating veterinarians recorded outcome and the 6 characteristics of the hospitalized foals that were associated with survival in the original multivariable model. These included age group, presence of a suckle reflex, ability to stand, WBC count, serum creatinine concentration, and anion gap. Participating veterinarians were also asked to record their initial estimate of the probability of survival and expected outcome (eg, whether they thought the foal would survive or die) and to record the final outcome as follows: died, euthanized because of impending death (in the opinion of the clinician), euthanized for reasons other than impending death, or discharged alive. Data from foals that were euthanized for reasons other than impending death were removed from the analyses. Data from the remaining foals were used to validate the original predictive model.

Sensitivity, specificity, positive and negative predictive values, and accuracy of the model to predict survival in this population of patients were determined. The percentage of agreement beyond chance between the clinician and the model was expressed as the kappa coefficient (k), and the McNemar test was used to evaluate the significance of agreement beyond that expected by chance alone. The probability of survival as estimated by the clinician was compared with the probability of survival based on output from the model by use of the clinician's estimate as the pretest probability of survival and LRs generated from the model to estimate the posttest probability of survival. The difference in the probability of survival as estimated by the clinician and that generated by use of the clinician's estimate as the pretest probability of survival, combined with the LRs generated from the model, was tested for significance by use of a paired signed rank test.^f Median differences in probability estimates for foals that survived and nonsurvivors were reported.

Data from both studies were entered directly into a computerized database file.^g After all data were entered, the database file was imported into a separate file for statistical analyses.^h For final analyses, P < 0.05 was considered significant.

Results

For the retrospective portion of the study, medical records were evaluated for 1,050 foals; records of 1,020 foals met inclusion criteria. One hundred ten of 1,020 foals were euthanized, so records of 910 foals were eligible for construction of the model. Variables included in the final model were age group, presence or absence of a suckle reflex, ability to stand, WBC count, serum creatinine concentration, and anion gap. Complete data on all 6 variables included in the final model were available for 577 of 910 (63%) foals, of which 479 survived and 98 died. Survival rate for this population was 83%.

Of the 577 foals with complete data, 371 (64%) were treated at HEMI, 187 (32%) were treated at RREH, and 19 (3%) were treated at VTH-CVM-UT. Survival rates among the 3 hospitals were not significantly (P = 0.15) different and were 82% for HEMI, 86% for RREH, and 68% for VTH-CVM-UT. Median ages of foals were also not significantly (P = 0.38) different among hospitals. Results of the univariate analyses to identify historical, physical examination, and laboratory findings associated with survival in foals were tabulated (Tables 1 and 2). Surviving foals were more likely than nonsurvivors to have a higher temperature, be older, have had previous treatment for their illness, have had a longer duration of clinical signs prior to hospitalization, and have a suckle reflex. Surviving foals were less likely than nonsurvivors to be assigned to receive critical care, be premature, be associated with dystocia or caesarian section, or be recumbent. Surviving foals also had significantly higher mean values than nonsurvivors for WBC count, total neutrophil count, neutrophil-to-lymphocyte ratio, blood glucose concentration, and proportion with IgG concentration ≥ 800 mg/100 mL, whereas mean values for anion gap and serum creatinine concentration were significantly lower. There was a significant association among foals < 48 hours old that lacked a suckle reflex with FPT (odds ratio, 2.8 [95% confidence interval, 1.5 to 5.2]).

Variables that were significant in the univariate analysis were entered into a logistic regression model in a forward stepwise manner. Variables retained in the final model included age group, WBC count, ability to stand, presence of a suckle reflex, anion gap, and serum creatinine concentration (Table 3). The final model had an R² value of 0.52, a P value of 0.45 associated with the Hosmer-Lemeshow goodness-of-fit test, and a −2 log likelihood fit test value of 312, indicating a good fit of the model to the data. The sensitivity and specificity to predict the survival or nonsurvival at different cut-points (probabilities) that were generated by the model were determined (Figure 1). By use of a cutpoint of 0.64877, the sensitivity of the model was 92% and specificity was 74%. Positive and negative predictive values for this population of foals were 95% and 65%, respectively (Table 4). Accuracy of the model ranged from 33% to 89%, and the choice of a cut-point to distinguish foals that survive from nonsurvivors was made to maximize accuracy.

Receiver operator characteristic curve illustrating the sensitivity and specificity at various cut-points for a multivariable logistic regression model to predict survival in hospitalized foals (n = 577) ≤ 7 days of age.

Citation: Journal of the American Veterinary Medical Association 228, 11; 10.2460/javma.228.11.1748

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Receiver operator characteristic curve illustrating the sensitivity and specificity at various cut-points for a multivariable logistic regression model to predict survival in hospitalized foals (n = 577) ≤ 7 days of age.

Citation: Journal of the American Veterinary Medical Association 228, 11; 10.2460/javma.228.11.1748

• Download Figure
• Download figure as PowerPoint slide

Receiver operator characteristic curve illustrating the sensitivity and specificity at various cut-points for a multivariable logistic regression model to predict survival in hospitalized foals (n = 577) ≤ 7 days of age.

Citation: Journal of the American Veterinary Medical Association 228, 11; 10.2460/javma.228.11.1748

• Download Figure
• Download figure as PowerPoint slide

Validation of the model was based on data for 163 foals from 5 hospitals. When applied to the new populations, the model was able to accurately predict survival in 90% of foals (sensitivity) and predict nonsurvival in 46% of foals (specificity). The proportion of foals that survived was 137 of 163 (84%), and the positive and negative predictive values were 90% and 46%, respectively. The proportions of foals that survived were similar among the 5 hospitals (P = 0.7). Data on the clinician's prediction as to whether the foal would survive were available for 123 of 163 (75%) foals and were compared with the prediction based on the model. The accuracies of the clinician and model were similar (83% and 81%, respectively [P = 0.8]), but the predictions were not well correlated (k = 0.29; P = 0.07).

When the equine clinician's initial estimate of survival was combined with output from the model, estimates of the probability of survival in foals that were discharged alive increased by a median probability of 12% (P < 0.001) and estimates of survival for those foals that died had decreased by a median probability of 9%, which was not significant (P = 0.7). Likelihood ratios were calculated for various ranges of probabilities generated by the retrospective model on the population of foals from HEMI, RREH, and VTH-CVM-UT (Table 5).

Discussion

The accuracy of the model was determined prospectively by predicting survival in foals subsequently admitted to 5 hospitals; however, the utility of the model was based on the ability to combine information from the clinician's initial assessment with data from observations of 577 foals to improve the accuracy of prognostic information given to the client early in the course of hospitalization. When both sources of information were combined, there was a modest but significant increase in the accuracy of the estimate of probability of survival among foals that survived and a nonsignificant decrease in the accuracy of the estimate of probability of nonsurvival among foals that died.

Survival rates of 83% and 84% were observed in the sample of foals used to develop the predictive model and the sample used to validate the model, respectively. Median age of foals in the retrospective and prospective populations were also similar. This suggested that the sample of foals used to construct the model was representative of the general population of hospitalized foals from veterinary teaching hospitals and private equine hospitals. In addition, there were no significant differences in the proportions of foals that survived among the 5 hospitals that enrolled foals in the prospective study. The survival rate for study foals was higher than that found in other published studies.^6,7 In the development of 2 previous models designed to predict survival in foals admitted to neonatal ICUs, the reported survival rates were 69% and 66% for the sample of foals used to create the models and 76% and 50% for the sample of foals used to validate the models.^6,7 The survival rate in a sample of foals used to develop a model to predict survival among septicemic foals was 45%.¹ Given the worst-case scenario—that all foals that were euthanized would have died in both the retrospective and prospective populations—survival rates would have been 73% and 74%, respectively. These values are still higher than those reported for all but one of the groups mentioned in prior studies. Foals admitted to ICUs and septicemic foals represent subpopulations of hospitalized neonates selected because of severe disease. The differences in survival rates among these populations and the foals in this study can be explained by the fact that foals in this study were unselected and therefore represented a more complete spectrum from mild to severe illness.

Survival models in general are better at predicting outcome when validated prospectively with foals at the same hospital from which the model was created.⁶ In a previous study,⁶ validation of a foal survival model on a new group of foals from the same teaching hospital ICU resulted in a sensitivity of 83% and specificity of 87%. However, when the model was applied to a group of foals from a second teaching hospital ICU, the sensitivity remained at 83% but the specificity was reduced to 44%.⁶ In another study,⁷ validation of the model was performed by use of a new group of foals from the same university ICU; the sensitivity decreased from 92% to 83%, and the specificity decreased from 100% to 40%. One explanation for the increased variability in specificity (ability of the model to accurately classify nonsurvivors) when the model was validated with a different population of foals may be the inclusion of foals that were euthanized because of poor prognosis, leading to the possibility of misclassification with regard to survival. In addition, a larger proportion of foals usually survive, resulting in a smaller sample size of nonsurvivors from which to estimate the model specificity. We were able to validate the model with a group of foals that were admitted to 4 veterinary teaching hospitals and 1 private veterinary hospital distinct from the 3 hospitals used to create the model; however, only 26 of 163 (16%) of these foals were either euthanized because of impending death or died and therefore could be used to estimate specificity. The positive predictive value of the model in the retrospective population (95%) was similar to that in the prospective population (90%); however, the negative predictive value decreased from 65% to 46%. A probability of survival that is ≥ 90% represents valuable information that can be communicated to a client struggling with the decision of whether to embark on an intensive and costly treatment program. Just as important is the substantial (46%) probability of nonsurvival associated with a negative test result. The results of the prospective study suggested that the model was robust, and the results are likely to be applicable to foals admitted to private equine hospitals as well as veterinary teaching hospitals.

When adjusted for other factors in the model, foals that survived were more likely to have a suckle reflex. Absence of a suckle reflex may be caused by systemic illness or injury and result in inadequate intake of colostrum. If the illness or injury occurs in utero or within a few hours after birth, FPT may occur and decrease the foal's resistance to infection. ⁹ Many of the foals in the present study were young; 383 of 577 (66%) were < 48 hours old, and there was a significant association with FPT among foals < 48 hours of age that lacked a suckle reflex. However, when forced into the model, FPT was not significant and did not improve the fit of the model to the data. A study¹⁰ of mortality rate in an intensively managed mare herd concluded that a main factor in foal survival was FPT. Contrary to findings in the present study, findings of 2 of 3 prior studies, 1 on survival of septic foals¹ and the other on foals admitted to an ICU,⁶ found no association between suckle reflex and survival in univariate analyses and this variable was not included in the final models. The ages of foals in the 2 studies were < 4 weeks and < 14 days. If the suckle reflex becomes suppressed or absent after 2 days of age, foals are less likely to develop FPT, and lack of this reflex after 2 days is more likely to represent severe neurologic disease or circulatory shock. The third study⁷ of foals < 10 days of age admitted to an ICU did not include suckle reflex in the data analysis.

The factor most strongly associated with survival in study foals was the ability to stand on admission. This association has been reported in previous studies, suggesting that events leading to recumbency are related to survival. Foals with neonatal maladjustment syndrome that had a normal gestational length and delivery and were able to stand and walk had a much better (up to 80%) prognosis.¹¹ In septic foals < 4 weeks of age, survival was greater in foals standing on admission to the hospital than those unable to stand.¹ Prolonged recumbency may be related to the severity of illness and shock in which survival is likely to be lower. Dependent atelectasis may occur after prolonged recumbency because of fluid that infiltrates the pulmonary interstitium.¹² A recumbent foal that is ≤ 24 hours old and has been unable to nurse is subject to FPT and nutritional deprivation, which can further weaken the foal and suppress the immune system.

Median serum creatinine concentration in study foals that survived was 2.0 mg/dL (interquartile range, 1.4 to 3.6 mg/dL) and in nonsurvivors was 4.2 mg/dL (interquartile range, 2.7 to 6.6 mg/dL). In foals < 4 weeks of age with radiographic evidence of pulmonary disease,⁵ foals that died were 4.9 times as likely to have serum creatinine concentrations > 1.7 mg/dL, and surviving foals < 14 days of age admitted to an ICU⁶ had a lower mean creatinine concentration (2.79 mg/dL) than those that died (4.60 mg/dL); however, in the latter study, serum creatinine concentration did not retain significance and was subsequently removed from the final multivariable model. In separate studies of septic foals¹ and those admitted to an ICU,⁷ creatinine concentration was measured but did not reveal significance in the univariate analysis. In the latter 3 studies, foals were selected for severe disease (eg, admission to an ICU or sepsis) and were older than foals in the present study. Excessive serum creatinine concentration in a neonatal foal may be caused by muscle atrophy; renal disease; dehydration; or problems with the mare, including high creatinine concentration in mare serum or inadequate placental function.¹³ Less common conditions associated with high creatinine concentration include caesarian section, placentitis, induced parturtion, and premature placental separation. Median age for foals in the present retrospective case series was 1 day, at which time complications involving parturition are more likely to be observed. The inconsistencies in the association between serum creatinine concentration and survival among previous studies and the present study may be related to differences in the age of foals and the differing clinical criteria for inclusion in each of the studies. High serum creatinine concentration appears to be a better marker to separate foals ≤ 7 days of age with severe metabolic disease, and thus less likely to survive, from those with mild or moderate disease in an unselected population.

The proportion of foals that survived after hospitalization was directly related to increasing WBC, as designated by ordinal categories from 1 to 3. Univariate analyses in 3 prior studies^1,6,7 revealed that WBC counts were consistently higher among survivors than nonsurvivors. In the present study, surviving foals had a median WBC count of 7.4 × 10³ cells/mL and nonsurvivors had 3.3 × 10³ cells/mL. Despite substantial differences in WBC counts reported in the first 2 of the previous studies, a significant difference was only attained in the second study.⁶ Lack of significance in the first study¹ may have been related to the small sample size and variability associated with WBC counts. In the third study,⁷ mean WBC counts were high in survivors and nonsurvivors. In that study, survival rate was 66% (n = 56), versus 83% (577) in the present study. White blood cell counts among survivors and nonsurvivors in the third study were higher than those of survivors in the 2 prior studies and the present study. This suggests that the health-related events of foals admitted to the ICU in the third study differed substantially.

Two previous studies included anion gap as a factor in multivariable models to predict survival in foals with radiographic evidence of pulmonary disease⁵ and foals < 10 days of age admitted to an ICU.⁷ Of 2 previous studies of adult horses with colic, 1 study¹⁴ did not include anion gap in the final multivariate analysis; however, in the second study,¹⁵ increasing values of anion gap correlated well with a decrease in survival rate. High anion gap is associated with shock, diabetic ketoacidosis, and renal failure in horses¹⁶ and is an indicator of metabolic acidosis in foals.¹⁷ Metabolic acidosis is a common condition in ill neonatal foals¹⁸ and is most commonly seen when tissue perfusion and oxygen delivery are decreased. Causes of decreased tissue oxygen delivery include low cardiac output secondary to sepsis and asphyxia, pulmonary disease with severe hypoxemia, and reduced oxygen carrying capacity of the blood (severe anemia).¹⁸

The logistic regression model expresses the probability of survival on a continuous scale from 0% to 100%. Use of a single cut-point to describe the sensitivity and specificity of the model to predict survival in a hospitalized neonate can result in loss of information and distortion.¹⁹ Use of a single cut-point carries with it the assumption that all foals given a probability of > 0.64877 on the basis of output from the model have the same probability of survival, and similarly, all foals with a probability of ≤ 0.64877 are given the same, albeit different, probability of survival. Thus, if only a single cut-point of > 0.64877 was assigned to distinguish survivors and nonsurvivors, a foal with a given probability of survival of 0.99 (99%) on the basis of output from the model would be given the same probability of survival as all other foals with values ranging from 0.65 to 1. The LR is a ratio of the proportion of foals that survived, given that the output from the model lies within a narrow range of probabilities, compared with the proportion of nonsurvivors given the same narrow range of values. By ranking foals from the retrospective portion of the study in order of probability of survival and dividing them into subgroups on the basis of a narrow range of probabilities, more information contained in the data can be utilized. Clinicians can combine their estimate of the probability of survival with observations of foals in the present study, not on the basis of a single cut-point, but rather on whether the model-generated probability for survival falls within a narrow range of values along the probability distribution.

Data for creation of the model were collected retrospectively and are not as likely to be as complete or accurate as if collected for the purpose of the study. Specific information on the reason for euthanasia was not in the medical records extracted retrospectively; therefore, these foals were excluded from analyses. In addition, it was not possible to document with certainty that foals had not received oxygen supplementation or fluid therapy prior to obtaining samples for laboratory analysis. The population of foals used to create the model could have been biased toward those with clinical signs that suggested that certain tests were performed. However, when the median age and survival rate of foals that were included in the retrospective portion of the study were compared with the sample of foals that were not euthanized but were excluded from the study because of lack of complete data, they were not significantly different. Foals that were euthanized in the prospective study were included in the analysis only if they were euthanized because of impending death. The model was accurate when validated on an unselected population of foals admitted to equine hospitals distinct from those used to generate the model. These observations suggest that selection bias did not play a role in the results and indicate that the study patients are likely to be representative of all hospitalized foals. Because of the low negative predictive value of the model when applied to the prospective population of foals, it alone should not be used as the sole criterion on which to base a decision for euthanasia.

However, knowledge that a foal has a 46% probability of nonsurvival should be included in the decisionmaking process. The possibility of confounding exists with all epidemiologic studies. This issue was discussed in a prior report,⁶ and illness in the mare, serum IgG concentration, and prematurity were mentioned as possible confounding factors. In the study reported here, all were forced into the final model, none were significant, and all reduced the fit of the model to the data.

The model has been incorporated into a spreadsheetⁱ that is available upon request from the corresponding author. Practitioners may use it to combine their clinical impression of survival with the quantitative experience based on observations of a large sample of foals, giving a more accurate prediction of probability of survival during the first few hours of hospitalization. Field use of the model is problematic; if test results are not immediately available, a delay in the decision to refer to a hospital may affect prognosis, and clinical and laboratory characteristics measured at an earlier stage of disease may affect the accuracy of the model. Use of the model in an equine hospital setting can assist the client in making an informed decision regarding patient management and result in more efficient use of personnel and medical and surgical resources within the hospital.