Abstract
Objective—To determine the accuracy of neurologic data, survey radiographic results, or both for localization of the site of thoracolumbar intervertebral disk herniation in dogs.
Sample—338 dogs with surgically confirmed intervertebral disk herniation from disk spaces T10–11 to L6–7.
Procedures—Medical records and archived survey radiographs were reviewed for each case. Data were analyzed with multivariable logistic regression models. Three models were fit to develop subsets of the data consisting of survey radiographic data, neurologic examination data, and a combination of survey radiographic and neurologic examination data. The resulting models were validated by evaluating predictive performance against a validation subset of the data.
Results—Models incorporating survey radiographic data and a combination of survey radiographic and neurologic data had similar predictive ability and performed better than the model based solely on neurologic data but resulted in substantial errors in predictions.
Conclusions and Clinical Relevance—A combination of neurologic examination data as recorded in the medical records and radiographic data did not enhance predictive performance of multivariable logistic regression models over models limited to radiographic data. Neurologic and radiographic findings should not be used to completely exclude areas in an abnormal spinal cord region from further evaluation with advanced imaging.
Advanced imaging techniques such as myelography, CT, or MRI for localization of disk herniation are widely accepted as the standard of care for dogs evaluated for possible thoracolumbar intervertebral disk herniation.1–3 The accuracy of these advanced imaging techniques has led to decreased reliance on more basic evaluation techniques such as neurologic examination and survey radiography. However, there are clinical situations in which the more basic techniques could potentially assist in patient management, because the neurologic examination and survey radiographs are recommended for the initial diagnostic evaluation for cases of suspected spinal cord disease.4 Accurate neurologic and survey radiographic examination data could provide a triage mechanism, allowing veterinarians to rank the relative importance of different areas of the vertebral column or spinal cord for advanced imaging and treatment.
The reported accuracy of survey radiography for the diagnosis of thoracolumbar intervertebral disk disease in dogs varies widely, ranging from 51% to 98%.1–3,5–11 More recent studies3,7 have typically found lower accuracies within this range (approx 60%), although a study12 comparing CT, myelography, and survey radiography found an extremely high accuracy of 94.7% for survey radiography. The importance of the neurologic examination for localization of the lesion to different spinal cord regions is not contested,13–15 particularly because survey radiographs provide no information as to the side (right vs left) of a lesion. However, the usefulness of the neurologic examination results for localization of disk herniation to specific disk spaces is less clear. Although neither radiographic nor neurologic localization is sufficiently accurate for detecting the site of disk herniation, a combination could potentially provide information to guide follow-up imaging. Statistical models incorporating the radiographic and neurologic information could be used to predict the site of disk herniation to determine the relevance of these data in guiding follow-up imaging.
The goals of the study reported here were to retrospectively assess the accuracy of commonly used radiographic and neurologic findings for predicting the site of intervertebral disk herniation in the thoracolumbar portion of the vertebral column and to determine by use of logistic regression models whether a model that uses a combination of radiographic and neurologic findings has better predictive performance than models that use neurologic or radiographic examination data alone. The hypothesis was that a combination model would yield better results than models that use neurologic or radiographic findings alone.
Materials and Methods
Sample—Electronic medical records of cases at the University of Minnesota Veterinary Medical Center between January 2000 and January 2009 were obtained. For the purposes of this study, the area of interest in the thoracolumbar region of the vertebral column was defined as the 10 disk spaces from T10–11 to L6–7. Cases were selected for review if surgery was performed in the thoracolumbar region of the vertebral column between T10–11 and L6–7 and if the following criteria were met: the surgical report indicated disk material recovery or the attending clinician's final diagnosis was intervertebral disk herniation or intervertebral disk disease, the surgeon entered into the medical record a specific site or sites as the origin of the herniated disk or disks, and adequate survey radiographs (either hard copy or digital images) consisting of lateral and ventrodorsal views were available from the time at which (ie, shortly before) surgery was performed.
Medical records review—Breed, sex and neuter status, and age were obtained from the medical records. The radiographic data were obtained by review of the archived survey radiographs by 2 board-certified radiologists (TM and DAF) who conferred with each other and came to a consensus. The radiologists were unaware of the neurologic examination findings or the results of subsequent tests and procedures; however, the radiologists were not unaware of the fact that these patients had thoracolumbar intervertebral disk disease. Each disk space was evaluated for narrowing of the intervertebral disk space, wedging of the intervertebral disk space, increased opacity of the intervertebral foramen, decreased size of the intervertebral foramen, collapse of the intervertebral facet joint, spondylosis deformans, mineralization of the disk in situ, and partial herniation of a mineralized disk. A partially herniated mineralized disk was defined as an incompletely mineralized disk that could be seen extruded into the intervertebral foramen. Digitally archived radiographs were reviewed with commercially available viewing software.a
The neurologic examination results extracted from the electronic medical records consisted of patellar reflex results, flexor reflex results, level of hyperesthesia, level of loss of CTR, and urinary bladder findings (turgidity and ease of expressibility). Complete documentation of the neurologic examination was not required for inclusion in the study.
The urinary bladder, patellar, and flexor reflexes were used, on the basis of the reflex arc assessed by each test, to determine whether disk herniation occurred in the spinal cord region (UMN vs LMN) indicated by the neurologic test result. The UMN or LMN regions were defined on the basis of neuroanatomic references16–20 to relate the affected spinal cord segments to disk spaces. For the purposes of this study, a distended bladder that was difficult to express indicated a potential disk herniation from T10–11 to L3–4 (UMN bladder dysfunction), whereas an easily expressed bladder indicated a potential disk herniation from L4–5 to L6–7 (LMN bladder dysfunction). A UMN patellar reflex response, defined as an exaggerated response, was classified as indicating a potential spinal cord lesion in the vertebral column from T10–11 to L2–3, whereas an LMN patellar reflex response, defined as an absent or decreased response, indicated a lesion from L3–4 to L6–7. A UMN flexor reflex response was defined as an exaggerated response and interpreted as indicating a spinal cord lesion in the vertebral column from T10–11 to L3–4, whereas an LMN flexor reflex response was defined as an absent or decreased response and interpreted as indicating a spinal cord lesion in the vertebral column from L4–5 to L6–7. Loss of the CTR was defined as indicating a spinal cord lesion at the disk 2 spaces cranial to the transverse plane where the reflex was lost, in keeping with an accepted interpretation of this test.13,21 For this study, CTR loss at T10–11 and T11–12 was assigned to T10–11 because use of the interpretation of CTR loss as indicating a spinal cord lesion 2 disk spaces cranial to the transverse plane of reflex loss would have assigned the lesion to areas of the vertebral column that were not evaluated in this study. Hyperesthesia over the vertebral column was considered to indicate disk herniation directly beneath the hyperesthetic site. If the medical records indicated an affected area of the thoracolumbar portion of vertebral column as opposed to a single intervertebral disk site, the center of the area was entered into the data set as a single site. Dogs with multiple sites of disk herniation were included in the analyses for the urinary bladder findings and the patellar and flexor reflexes if all of the disk herniations affected either the UMN or LMN regions of the spinal cord but not both regions. If herniated disks were found affecting both the UMN and LMN regions of the spinal cord, the dog was excluded from these analyses. Radiographic abnormalities were considered to indicate a potential disk herniation at that level.
Radiographic abnormalities, loss of CTR, and hyperesthesia were defined as present or absent at each disk site. If CTR or hyperesthesia data were missing, neurologic abnormalities were classified as absent for the 10 disk spaces in that dog. The urinary bladder findings, flexor reflex results, and patellar reflex results for each dog were interpreted as constituting 1 of 3 possible categories: normal (reference range) results, missing (from the medical record) results, or nondiagnostic results; disk herniation in the UMN region of the spinal cord; or disk herniation in the LMN region of the spinal cord. For the patellar and flexor reflex data, the results from the left and right side were combined. If the results from 1 side were graded as normal or no results had been recorded, the results for that side were classified according to the result from the contralateral limb if the contralateral limb had a UMN or LMN finding. If the reflex result was recorded for both sides and the results were the same, that result was recorded. If the results from the opposite sides disagreed (indicating spinal cord lesions in both the UMN and LMN regions of the spinal cord), the case was classified in the normal, missing, or nondiagnostic category for that neurologic test result.
Statistical analysis—For cases with missing neurologic test results, the assumption was made that values were missing because the neurologic test result was normal and therefore not recorded. To test the effect of this strong assumption, sensitivity analysis was performed to compare the missing data assumed to indicate a negative (normal) result versus data assumed to be missing at random. The data assumed to be missing at random were imputed by use of the multivariate imputation by chained equations packageb in a software program.c The package specifies the multivariate imputation model on a variable-by-variable basis by a set of conditional densities, 1 for each incomplete variable. Starting from an initial imputation, imputations are drawn by iterating 10 to 20 times over the conditional densities.22 Although not fully Bayesian, the approach was used because there was no suitable multivariate distributional specification for the missing values.
An initial univariate logistic regression analysis was performed on all of the patient data to determine which variables had any association with the surgically confirmed results when considered alone. Because there was a hierarchic structure to the data set caused by the clustering of disks within dogs, a multilevel logistic regression model was fit to the development data set to determine whether there was a relevant random effects component among dogs. All 2-way interactions were tested for significance (P < 0.05). The data set was randomly separated into development (70% of the cases) and validation (30% of the cases) data sets by use of a random number generatorc to allow validation with a randomly selected subset of the data that was not used for development of the multivariable logistic regression models. Three models were fit to the development data set: a neurologic model limited to the neurologic variables, a radiographic model limited to the radiographic variables, and a full model that incorporated the neurologic and radiographic variables. The predictor variables consisted of the neurologic variables, radiographic variables, and disk location (a varying intercept term for the 10 disks considered in this study). Bayesian model averaging was used for the modeling process. In BMA, the posterior probabilities of all possible models that result from different combinations of the predictor variables are calculated. The posterior probability of each model reflects the probability that, given the data, the model in question is the true model. The final model parameters of interest, such as regression coefficients, are obtained by a weighted mean of the results of the most probable models. The weighting factor for this value is the posterior probability of the models. Statistical inferences and predictions are based on the weighted mean results of all models with posterior probabilities greater than a preestablished threshold.23,24 A standard threshold was used, which excluded all models that were < 1/20th the probability of the model with the highest posterior probability24
Three quantities are usually provided when reporting BMA results for a regression analysis.23 Mean (β|D) represents the posterior mean of the regression coefficient, which is the weighted mean result of all the models that incorporate the variable in question. The weighting is based on the posterior probability of each model, which incorporates the variable. The SD(β|D) is the SD of the posterior mean for the regression coefficient. The Pr(i≠0|D) represents the summed posterior probability of all models that incorporate the predictor variable in question and can be viewed as the strength of the evidence supporting the predictor variable as associated with the outcome. A reported standard for interpreting Pr(βi≠0|D) indicates that a value < 50% suggests no evidence for an association, 50% to 75% suggests weak evidence, > 75% to 95% indicates positive evidence, and > 95% indicates strong evidence.24 The models were used to make predictions with the validation data set, and the accuracies of the predictions were assessed with ROC curves. The AUC provides an overall measure of predictive accuracy.
Because ROC curves provided an overall assessment of model prediction across all decision thresholds, a more direct, clinically interpretable measure of model prediction was obtained by assigning the disk in each dog with the highest predicted probability of herniation as the model's predicted site of herniation. Predictions were made with the validation data set. Some dogs in the validation data set were expected to have > 1 site of disk herniation, and each model could only predict a single site of herniation per dog.
All data were recorded with a commercially available spreadsheet program.d Data analysis and statistical model fitting were performed with a publicly available statistical program,c with add-in packages for BMA,e logistic regression analysis,f and ROC curve plots.25,26,g,h For all comparisons, values of P < 0.05 were considered significant.
Results
Three hundred thirty-eight dogs met the inclusion criteria. Dachshund was the most common breed, with 170 Miniature and Standard Dachshunds, followed by Cocker Spaniel (n = 21), Basset Hound (17), Bichon Frise (12), Labrador Retriever (12), Shi Tzu (10), Lhasa Apso (7), and Pekingese (7). Thirty-four other breeds were represented with ≤ 5 dogs each. There were 13 mixed-breed dogs and a single dog of unknown breed. There were 168 castrated males, 131 spayed females, 26 sexually intact males, and 13 sexually intact females. The mean ± SD age was 6.3 ± 2.7 years, with a range from 1.9 to 16.8 years. Three hundred eighty-two sites of disk herniation were surgically confirmed in the 338 dogs, with 1 dog having 4 sites of herniation, 3 dogs having 3 sites of herniation, 35 dogs having 2 sites of herniation, and the remaining 299 dogs having a single site of herniation. The numbers of disk herniations at each site were as follows: T10–11 (n = 3), T11–12 (58), T12–13 (103), T13-L1 (101), L1–2 (48), L2–3 (36), L3–4 (22), L4–5 (9), L5–6 (2), and L6–7 (0).
Results of the neurologic and radiographic examination findings and the surgically confirmed sites of disk herniation were obtained, and univariate logistic regression analyses were performed (excluding cases with missing values) for the urinary bladder, patellar reflex, and flexor reflex (Table 1), hyperesthesia and CTR (Table 2), and radiographic findings (Table 3).
Results of univariate logistic regression analysis of the associations between results of examination of the urinary bladder, patellar reflex, and flexor reflex and surgically confirmed involvement of the UMN or LMN region of the spinal cord in 338 dogs with intervertebral disk herniation.
| Surgical findings (No. of dogs) | |||||
|---|---|---|---|---|---|
| Variable | UMN | LMN | OR | 95% CI | P value |
| Urinary bladder | |||||
| Normal | 61 | 4 | Referent | Referent | |
| UMN | 4 | 0 | NA | NA | 0.99 |
| LMN | 24 | 4 | 2.54 | 0.59–10.99 | 0.21 |
| Patellar reflex | |||||
| Normal | 54 | 3 | Referent | Referent | |
| UMN | 112 | 7 | 1.12 | 0.28–24.36 | 0.87 |
| LMN | 10 | 2 | 3.60 | 0.53–24.36 | 0.19 |
| Flexor reflex | |||||
| Normal | 66 | 4 | Referent | Referent | |
| LMN | 10 | 6 | 9.90 | 2.37–41.34 | 0.001 |
NA = Not applicable (complete separation occurred, leading to lack of convergence of the maximum likelihood estimate algorithm).
Results of univariate logistic regression analysis of the associations between hyperesthesia and negative results of the CTR and their anatomic relationship to a surgically confirmed site (at the confirmed site or within 1 or 2 disk spaces of the confirmed site) of intervertebral disk herniation in a subset of the same dogs as in Table 1.
| Neurologic test | Not availablea (No. of dogs) | Normal test results (No. of dogs) | At surgically confirmed site (proportion of total) | Within 1 disk space (proportion of total) | Within 2 disk spaces (proportion of total) | ORb | 95% CI | P value |
|---|---|---|---|---|---|---|---|---|
| Hyperesthesia | 129 | 46 | 52/163 (0.319) | 125/163 (0.763) | 154/163 (0.945) | 4.60 | 3.18–6.64 | < 0.001 |
| CTR | 247 | 11 | 32/80 (0.400) | 60/80 (0.750) | 70/80 (0.875) | 6.31 | 3.80–10.48 | < 0.001 |
Test not performed or location not recorded.
Odds ratios defined in terms of odds of disk herniation in the site of hyperesthesia or 2 disk spaces cranial to the site of CTR loss versus the odds of disk herniation in disk spaces without hyperesthesia or CTR loss, respectively.
Results of univariate logistic regression analysis of the associations between radiographic abnormalities and surgically confirmed sites of disk herniation in the same dogs as in Table 1.
| Surgically confirmed herniation at site (No. of disks) | |||||
|---|---|---|---|---|---|
| Radiographic abnormality | Yes | No | OR | 95% CI | P value |
| Narrowed intervertebral disk space | 14.13 | 11.11–17.95 | < 0.001 | ||
| Present | 260 | 393 | |||
| Absent | 122 | 2,605 | |||
| Wedged intervertebral disk space | 21.20 | 14.11–31.86 | < 0.001 | ||
| Present | 80 | 37 | |||
| Absent | 302 | 2,961 | |||
| Small intervertebral foramen | 43.01 | 31.02–59.64 | < 0.001 | ||
| Present | 177 | 59 | |||
| Absent | 205 | 2,939 | |||
| Opaque intervertebral foramen | 22.40 | 16.64–30.17 | < 0.001 | ||
| Present | 151 | 85 | |||
| Absent | 231 | 2,913 | |||
| Facet space narrowing | 21.58 | 16.25–28.65 | < 0.001 | ||
| Present | 164 | 101 | |||
| Absent | 218 | 2,897 | |||
| Spondylosis deformans | 1.64 | 0.97–2.75 | 0.06 | ||
| Present | 18 | 88 | |||
| Absent | 364 | 2,910 | |||
| In situ mineralized disk | 0.83 | 0.58–1.17 | 0.28 | ||
| Present | 39 | 363 | |||
| Absent | 343 | 2,635 | |||
| Partially (ruptured) mineralized disk | 27.46 | 17.30–43.59 | < 0.001 | ||
| Present | 74 | 26 | |||
| Absent | 308 | 2,972 | |||
Odds ratios defined in terms of odds of intervertebral disk herniation at the site of a radiographic abnormality versus the odds of disk herniation at the disk site without a radiographic abnormality. Ten disks were evaluated in each of the 338 dogs (total, 3,380 disks).
Urinary bladder findings were recorded for 97 dogs, with findings considered normal in 65; results were missing for 241 dogs. Eight of the 32 dogs with abnormal urinary bladder findings were correctly classified as having involvement of the UMN or LMN areas. Patellar reflex results were available for 188 dogs, with results considered normal in 57 dogs; results were missing for 150 dogs. One hundred fourteen of 131 dogs were correctly classified as having involvement of the UMN or LMN areas, with the majority (112 dogs) of the correct results in the UMN category. Flexor reflex results were recorded for 87 dogs, but 1 dog was excluded from the spinal cord region–specific analysis because of disk herniations in the UMN and LMN regions, leaving 86 dogs in the analysis with results considered normal in 70 dogs; results were missing for 252 dogs (Table 1). Six of 16 dogs with abnormal results were correctly classified as having involvement of the UMN or LMN areas. In no cases was a response to the flexor test considered hyperreflexic; therefore, this category (hyperreflexic flexor response) was removed from further analyses. Of the region-specific neurologic tests, the presence of a decreased or absent flexor reflex was significantly associated with disk herniation affecting the LMN region in a univariate analysis. Urinary bladder findings and patellar reflex findings were not significantly associated with disk herniation affecting the LMN region.
A neurologic examination for hyperesthesia was performed in 227 dogs (111 dogs had neurologic examination results with no mention of examination for hyperesthesia) and for the CTR in 80 dogs (Table 2). Forty-six dogs had neurologic examination results indicating no hyperesthesia along the thoracolumbar portion of the vertebral column, and 18 dogs had neurologic examination results indicating hyperesthesia but with no indication of the location. Of the 163 dogs with hyperesthesia examination results that indicated an affected site, 52 had hyperesthesia over the surgically confirmed site of disk herniation. Although not specifically analyzed, 125 of the 163 (77%) dogs had hyperesthesia within 1 disk space of a surgically confirmed site of disk herniation and 154 of the 163 (94%) dogs had hyperesthesia within 2 disk spaces of the site of disk herniation. In 80 dogs, the CTR reflex was absent, indicating the potential location of an affected disk or region of the thoracolumbar portion of the vertebral column. Of the remaining 258 dogs, 11 had neurologic examination results indicating no CTR loss, 12 had results indicating loss of the CTR but no indication of the location along the vertebral column at which CTR loss occurred, and 235 had neurologic examination results with no mention of examination for CTR loss. Both CTR loss and hyperesthesia were significantly associated with the site of disk herniation in a univariate analysis with missing, unspecified, and normal results classified as a single baseline category.
Sensitivity analysis (univariate logistic regression) of the data comparing missing values assumed to represent negative (normal) results versus missing data assumed to be missing at random and imputed revealed no substantial differences between the 2 data sets. Therefore, subsequent analyses were performed with the assumption that missing data represented negative (normal) results.
For the radiographic variables, there were no missing values because survey radiographs were required as an inclusion criterion. A narrowed intervertebral disk space, wedged intervertebral disk space, small intervertebral foramen, opaque intervertebral foramen, collapsed facet joint, and a partially herniated disk were significantly associated with confirmed sites of disk herniation, whereas spondylosis deformans and mineralized disk in situ were not significantly associated with herniation (Table 3).
There were no estimable between-dog random effects, indicating that there was no detectable difference among dogs with respect to the probability of disk herniation. Inclusion of dog random effects did not improve model fit. Therefore, dog-level random effects models were not considered for the remaining analyses. No significant 2-way interactions were present. The results of the BMA multivariable model of the neurologic variables (Table 4) indicated that the level of CTR loss and hyperesthesia was associated with the site of disk herniation in a multivariate analysis, with posterior probabilities of 100% and 78.5% respectively. The results of the urinary bladder evaluation, patellar reflex, and flexor reflex were not associated with the site of disk herniation. For the radiographic model (Table 5), a narrowed intervertebral disk space, opaque intervertebral foramen, small intervertebral foramen, and partial ruptured disk were strongly associated with the site of disk herniation, whereas a narrowed facet joint was associated with the site of disk herniation. A wedged intervertebral disk space, spondylosis deformans, and a mineralized disk in situ were not associated with disk herniation.
Results of BMA multivariable logistic regression for a neurologic model with outcome defined as the site of intervertebral disk herniation in dogs.
| Neurologic abnormality | Mean(β|D) | SD(β|D) | OR (95% CI) | Pr(βi≠0|D) |
|---|---|---|---|---|
| Bladder | ||||
| LMN | 0.0 | — | — | 0% |
| UMN | 0.0 | — | — | 0% |
| LMN patellar reflex | 0.0 | — | — | 0% |
| UMN patellar reflex | 0.0 | — | — | 0% |
| LMN flexor reflex | 0.069 | 0.36 | 1.07 (0.53–2.17) | 4.7% |
| CTR loss | 1.20 | 0.29 | 3.32 (1.88–5.86) | 100% |
| Hyperesthesia | 0.61 | 0.38 | 1.84 (0.87–3.88) | 78.5% |
— = Results not available because variables had Pr(βi≠0|D) = 0%. Mean(β|D) = Posterior mean of the regression coefficient. Pr(β≠0|D) = Summed posterior probability of all models incorporating the predictor variable in question, which reflects the strength of evidence supporting an association between the predictor variable and the outcome (herniated disk). SD(β|D) = SD of the estimated regression coefficient.
Odds ratios are defined in terms of odds of disk herniation in the LMN regions of the spinal cord with the given neurologic finding versus the odds of disk herniation in the LMN regions of the spinal cord with baseline (normal) neurologic findings.
Results of BMA multivariable logistic regression for a radiographic model with outcome defined as the site of intervertebral disk herniation.
| Variable | Mean(β|D) | SD(β|D) | OR (95% CI) | Pr(βi≠0|D) |
|---|---|---|---|---|
| Narrowed intervertebral disk space | 1.68 | 0.24 | 5.37 (3.35–8.59) | 100% |
| Wedged intervertebral disk space | 0.0 | — | — | 0% |
| Opaque intervertebral foramen | 1.08 | 0.26 | 2.94 (1.77–4.90) | 100% |
| Small intervertebral foramen | 1.11 | 0.28 | 3.03 (1.75–5.25) | 100% |
| Facet space narrowing | 0.85 | 0.34 | 2.34 (1.20–4.56) | 93.2% |
| Spondylosis deformans | 0.056 | 0.23 | 1.06 (0.67–1.66) | 7.4% |
| In situ mineralized disk | 0.032 | 0.14 | 1.03 (0.78–1.36) | 6.9% |
| Partially (ruptured) mineralized disk | 2.01 | 0.43 | 7.46 (3.21–17.34) | 100% |
See Table 4 for key.
When the neurologic and radiographic variables were combined in the full model (Table 6), a narrowed intervertebral disk space, opaque intervertebral foramen, small intervertebral foramen, and a partially ruptured disk were still strongly association with the site of disk herniation. The level of CTR loss and a narrowed facet joint were associated with the site of disk herniation. The remaining variables were not associated with the site of disk herniation.
Results of BMA multivariable logistic regression for a full model combining radiographic and neurologic variables with outcome defined as the site of intervertebral disk herniation.
| Variable | Mean(β|D) | SD(β|D) | OR (95% CI) | Pr(βi≠0|D) |
|---|---|---|---|---|
| Urinary bladder | ||||
| LMN | 0.0 | — | — | 0% |
| UMN | 0.0 | — | — | 0% |
| LMN patellar reflex | 0.0 | — | — | 0% |
| UMN patellar reflex | 0.0 | — | — | 0% |
| LMN flexor reflex | 0.0 | — | — | 0% |
| CTR loss | 0.90 | 0.62 | 2.46 (0.73–8.29) | 74.5% |
| Hyperesthesia | 0.011 | 0.084 | 1.01 (0.86–1.19) | 2.5% |
| Narrowed intervertebral disk space | 1.70 | 0.25 | 5.47 (3.35–8.94) | 100% |
| Wedged intervertebral disk space | 0.0 | — | — | 0% |
| Opaque intervertebral foramen | 1.07 | 0.26 | 2.91 (1.75–4.85) | 100% |
| Small intervertebral foramen | 1.12 | 0.28 | 3.06 (1.77–5.31) | 100% |
| Facet space narrowing | 0.75 | 0.37 | 2.12 (1.03–4.37) | 87.8% |
| Spondylosis deformans | 0.049 | 0.22 | 1.05 (0.68–1.62) | 6.0% |
| In situ mineralized disk | 0.021 | 0.11 | 1.02 (0.82–1.27) | 4.5% |
| Partially (ruptured) mineralized disk | 2.06 | 0.44 | 7.85 (3.31–18.59) | 100% |
See Table 4 for key.
The ROC curves obtained with the models’ predictions for the validation data set (Figure 1) revealed that the neurologic model poorly predicted the outcome variable for the validation data set. The AUCs for the models were 0.829 (95% CI, 0.799 to 0.860) for the neurologic model, 0.898 (95% CI, 0.870 to 0.926) for the radiographic model, and 0.895 (95% CI, 0.864 to 0.925) for the full model, which indicated that there was a significant difference between the accuracies of the neurologic model versus the radiographic and full models. Model predictions tested by use of the validation data set and based on the disk location with the highest predicted probability of herniation for each dog were obtained (Table 7). The measures of accuracy were similar between the radiographic and full models but lower for the neurologic model. All 3 models had a substantial number of errors in prediction when tested with the validation data set.
Receiver operating characteristic curve plots for full model, radiographic model, and neurologic model predictions performed with a validation data set in a study of the accuracy of prediction of the site of thoracolumbar intervertebral disk herniation in 338 dogs.
Citation: American Journal of Veterinary Research 75, 3; 10.2460/ajvr.75.3.251
Accuracy of predictions made with a validation subset of the data by the neurologic, radiographic, and full models on the basis of the site of highest predicted probability of intervertebral disk herniation in 1,020 intervertebral disks in 102 dogs.
| Model | True positive | False positive | False negative | True negative | Sensitivity | Specificity | Positive predictive value | Negative predictive value |
|---|---|---|---|---|---|---|---|---|
| Neurologic | 34 | 68 | 91 | 827 | 0.272 | 0.924 | 0.333 | 0.901 |
| Radiographic | 76 | 26 | 49 | 869 | 0.608 | 0.971 | 0.745 | 0.947 |
| Full | 77 | 25 | 48 | 870 | 0.616 | 0.972 | 0.755 | 0.948 |
Total numbers of true-positive, false-positive, false-negative, and true-negative results were calculated from a validation data set that contained 102 dogs with 125 disk herniations in 1,020 thoracolumbar disks. Models were limited to a single positive prediction per dog; therefore, positive predictions totaled 102 and negative predictions totaled 1,018.
Discussion
Multivariable regression models estimate the association of an independent or predictor variable with the outcome or dependent variable while statistically adjusting for the other predictor variables in the model.27 For the study reported here, an attempt was made to statistically model the diagnostic process to objectively determine which variables were most strongly associated with surgically confirmed sites of disk herniation. For this purpose, a large retrospective clinical data set was used. Although affected by the limitations of the data set, this kind of analysis may allow more objective analysis of the diagnostic process than other methods. For this data set, CTR and hyperesthesia were the neurologic variables associated with the site of intervertebral disk herniation when all neurologic and radiologic variables were included in the full model. The remaining neurologic variables were not associated with the site of herniation on the basis of BMA analysis. Of the radiographic variables considered for this study, a narrowed intervertebral disk space, opaque intervertebral foramen, small intervertebral foramen, narrowed facet space, spondylosis deformans, and a partially herniated disk within the spinal canal were positively associated with the site of herniation. The neurologic findings had little influence on model prediction of a specific site of disk herniation.
When normal neurologic examination results were excluded, localization of the disk herniation to UMN versus LMN regions of the spinal cord based on the turgidity of the urinary bladder and ease of manual expression was correct in only 8 of 32 cases. The flexor reflex was correct in only 6 of 16 cases when the flexor reflex was assessed as decreased or absent and therefore suggestive of a lesion in the LMN region. Only the patellar reflex had a high level of accuracy, with 114 of 131 cases correctly classified when the patellar reflex was classified as hyperreflexic and therefore suggestive of a lesion in the UMN region. However, when a patellar reflex was assessed as decreased and therefore suggestive of a lesion in the LMN region, only 2 of 12 cases were correctly classified. In an earlier study, predictions of a lesion in the UMN versus LMN regions had 94% agreement with the surgical findings.5 It is not clear which of the neurologic tests were assessed in the earlier study; this assessment was most likely based on a combination of both the patellar and flexor reflexes. In assessing the strength of association of a test result with disease, one must consider inherent test characteristics such as sensitivity and specificity as well as extrinsic characteristics such as the prevalence of disk herniation in different areas of the vertebral column.28 It is also important to acknowledge the user dependence of a test, whether from neurologic examination or survey radiography, in that a test may work well for some and not for others. Disease prevalence will affect positive and negative predictive values; therefore, although certain neurologic examination results such as patellar hyperreflexia may be highly accurate, the contribution to predicting the site of disk herniation is low because of the extremely high relative occurrence of disk herniation in the UMN versus LMN regions of the spinal cord. Only 11 of the 382 disk herniations occurred in the LMN regions of the spinal cord, and this low frequency led to a low positive predictive value for any neurologic test result that indicated a lesion in the LMN region. However, the patellar and flexor reflexes could provide information on the laterality of a lesion, which was not evaluated in this study.
Hyperesthesia and CTR loss provided more localizing information than the other neurologic tests because these tests could indicate specific disk sites as potential sites of herniation. Hyperesthesia in this data set yielded approximately 32% accuracy, correctly indicating the specific site of disk herniation in 52 of 163 cases in the present study, compared with 34% correct localization reported previously.5 Allowing 2 disk spaces of error led to a correct classification proportion of 154 of 163 (95%), which was higher than the 64%5 and 70%6 previously reported. However, these results were based on dogs that had hyperesthesia, and 46 of the dogs had no hyperesthesia. Hyperesthesia was not associated with accurate localization of the lesion when analyzed in the full model. We believe this result was attributable to the high number of cases in which no hyperesthesia was found, the inaccuracy of hyperesthesia for lesion localization purposes when it is strictly interpreted as indicating the immediately underlying disk space, and classifying missing results as normal with respect to hyperesthesia. In a clinical setting, hyperesthesia would likely provide an indication as to which lesions were clinically important. However, the absence of hyperesthesia in 46 of the 209 (22%) cases with confirmed disk herniation indicated that apparently normal regions of the vertebral column cannot be excluded from further evaluation with advanced imaging. Loss of the CTR correctly indicated disk herniation 2 spaces cranial to the level of loss of the CTR in 32 of 80 (40%) cases, which was greater than the 32% accuracy in a previous report.5 In a more recent study,29 48.5% of the dogs had loss of the CTR 2 to 3 vertebrae caudal to the lesion. The level (vertebral) of loss of the CTR in our data set was more strongly associated with the site of herniation than were the remaining neurologic variables. However, this variable was only weakly associated with the site of herniation when all of the radiographic and neurologic variables were considered. As with hyperesthesia, we believe this result occurred because of the high proportion of cases with a normal CTR, the limitations imposed by strictly defining loss of the CTR as indicating a lesion 2 intervertebral disk spaces cranial to the level of the loss of the CTR, and the classification of missing results as indicating a normal CTR.
In this study, most of the radiographic signs of disk herniation were strongly associated with the site of disk herniation. However, the cross-tabulated data of radiographically abnormal sites versus surgically confirmed sites clearly revealed a high number of false-positive and false-negative predictions if the radiographic abnormalities were interpreted as indicating the site of disk herniation, indicating an unacceptable level of error, as in other reports.1–3,5,7–9 The finding of a narrowed disk space had the highest number of true-positive findings; however, there was a high number of false-positive findings. The positive predictive value calculated from the results was approximately 40% in this study, which was considerably lower than a previous report4 of 63% to 71%.
One goal of the study reported here was to determine whether combinations of objective neurologic and radiographic examination findings were more accurate than either type of data alone. The predictions from the radiographic model and full model (combining neurologic and radiographic findings) were nearly identical, which indicated that accuracy did not improve with the addition of neurologic data. Both models had a statistically higher predictive ability than the neurologic model when the AUCs for ROC curve plots were compared. However, this difference may not be clinically helpful because when the disk site with the highest predicted probability was set as the model's choice for site of disk herniation, a large number of false-positive and false-negative findings occurred in both models. A sensitivity of approximately 60% and positive predictive values of 74% to 75% were obtained for the radiographic and full models. The 62% rate of detection of the affected disks in the validation data set was similar to that of most previous reports.3,5,11 The model-based results were in agreement with the conclusions of authors of previous reports that advanced imaging is required to guide surgical intervention.
There were limitations of this study that must be considered when interpreting its results. The use of a retrospective data set led to limitations in the data set. A large number of values were missing for neurologic examination findings. Assuming that these missing values equated to a normal result could have weakened the association of neurologic findings with disk herniation. There may have been selection bias, in that cases with a complete neurologic examination may have been different (eg, more complex) than cases with a limited neurologic examination. These missing data accurately reflected the realities of clinical practice, where information may not be available to those interpreting the imaging studies, particularly if the surgeon or neurologist performing a surgical procedure did not initially examine the patient. The extraction of neurologic variables from the medical records of a tertiary referral institution also meant that there were a large number of interpreters of the neurologic examination, ranging in experience from interns and residents to board-certified surgeons and neurologists. Although the radiographic data were obtained by evaluation of archived radiographs by 2 board-certified veterinary radiologists, the neurologic data were obtained by clinicians with various levels of training and expertise. Different results may have been obtained if all the neurologic examinations were performed by a board-certified neurologist. There was also a lack of standardized reporting of the neurologic variables, which may have led to inaccurate representation of the neurologic examination. For example, the interpretation of hyperesthesia occurring at the thoracolumbar junction as hyperesthesia occurring at the T13-L1 disk space may have been a more specific interpretation of this result than was intended by the clinician. Classification of a large, easily expressed urinary bladder as indicating LMN bladder dysfunction may have been simplistic, considering that reflexive bladder emptying could also be interpreted as easy bladder expression.30 Further refinement of the neurologic findings would require a fully standardized neurologic examination, either as an institutional policy or as part of a prospective study. Defining the outcome on the basis of the results of surgical exploration could have resulted in incorrect classification of the outcome. Surgically unexplored disk sites were classified as negative for disk herniation, which may have been incorrect. Therefore, although the surgically confirmed herniated disks could be regarded as true-positive results, negative disk herniation classifications were actually based on the sensitivities of the follow-up imaging tests that classified those disks as not herniated. Radiographic and neurologic results indicating a lesion in a surgically unexplored site cannot be unequivocally classified as incorrect. Additionally, it was not possible to be certain that a surgically confirmed lesion was responsible for the clinical signs or that there had not been complicating factors such as myelomalacia that may have caused an incongruous neurologic finding. There is known inherent variability in the relationship between spinal cord segments and the vertebral bodies in dogs, associated with breed and size.16 Finally, to use the data set for statistical modeling, we were forced to make assumptions about what represented a correct or incorrect result for the neurologic examination results. Because the outcome variable was defined strictly in terms of longitudinal localization of disk herniation, the results from neurologic tests with bilateral results were combined. This could have led to misclassification of a neurologic test result as incorrect if the dog truly had asymmetric neurologic signs.
The use of the combination of objective neurologic and radiographic examination findings in the statistical models did not improve accuracy of prediction of the site of disk herniation, compared with other published values. In our opinion, the neurologic and radiographic findings could be used to evaluate different areas of the vertebral column once baseline advanced imaging is obtained but should not be used to completely exclude areas of an abnormal vertebral column region from evaluation with advanced imaging.
ABBREVIATIONS
| AUC | Area under the curve |
| BMA | Bayesian model averaging |
| CI | Confidence interval |
| CTR | Cutaneous trunci reflex |
| LMN | Lower motor neuron |
| ROC | Receiver operating characteristic |
| UMN | Upper motor neuron |
Carestream Health Inc, Rochester, NY.
mice: multivariate imputation by chained equations, R, version 1.0–4, R Development Core Team, Vienna, Austria. Available at: www.jstatsoft.org/v45/i03/. Accessed Jul 1, 2013.
R, version 1.0–4, R Development Core Team, Vienna, Austria.
MS Excel, Microsoft Corp, Redmond, Wash.
BMA: Bayesian model averaging, R, version 1.0–4, R Development Core Team, Vienna, Austria. Available at: CRAN.R-project.org/package=BMA. Accessed Jul 1, 2013.
brglm: bias reduction in binomial-response generalized linear models, R, version 1.0–4, R Development Core Team, Vienna, Austria. Available at: www.ucl.ac.uk/∼ucakiko/software.html. Accessed Jul 1, 2013.
ROCR, R, version 1.0–4, R Development Core Team, Vienna, Austria. Available at: rocr.bioinf.mpi-sb.mpg.de. Accessed Jul 1, 2013.
pROC, R Development Core Team, Vienna, Austria. Available at: www.biomedcentral.com/1471-2105/12/77. Accessed Jul 1, 2013.
References
1. Kirberger RM, Roose CJ, Lubbe AM. The radiological diagnosis of thoracolumbar disc disease in the Dachshund. Vet Radiol Ultrasound 1992; 33: 255–261.
2. Olby NJ, Dyce J, Houlton JEF. Correlation of plain radiographic and lumbar myelographic findings with surgical findings in thoracolumbar disc disease. J Small Anim Pract 1994; 35: 345–350.
3. Lamb CR, Nicholls A, Targett M, et al. Accuracy of survey radiographic diagnosis of intervertebral disc protrusion in dogs. Vet Radiol Ultrasound 2002; 43: 222–228.
4. Parent J. Clinical approach and lesion localization in patients with spinal diseases. Vet Clin Small Anim Pract 2010; 40:733–753.
5. Brown NO, Helphrey ML, Prata RG. Thoracolumbar disk disease in the dog: a retrospective analysis of 187 cases. J Am Anim Hosp Assoc 1977; 13: 665–672.
6. Sukhiani HR, Parent JM, Atilola MA, et al. Intervertebral disk disease in dogs with signs of back pain alone: 25 cases (1986–1993). J Am Vet Med Assoc 1996; 209: 1275–1279.
7. Schulz KS, Walker M, Moon M, et al. Correlation of clinical, radiographic, and surgical localization of intervertebral disc extrusion in small-breed dogs: a prospective study of 50 cases. Vet Surg 1998; 27: 105–111.
8. Macias C, McKee WM, May C, et al. Thoracolumbar disc disease in large dogs: a study of 99 cases. J Small Anim Pract 2002; 43: 439–446.
9. Rohdin C, Jeserevic J, Viitmaa R, et al. Prevalence of radiographic detectable intervertebral disc calcifications in Dachshunds surgically treated for disc extrusion. Acta Vet Scand 2010; 52:24.
10. McKee WM. A comparison of hemilaminectomy (with concomitant disc fenestration) and dorsal laminectomy for the treatment of thoracolumbar disc progrusion in dogs. Vet Rec 1992; 130: 296–300.
11. Lubbe AM, Kirberger RM, Verstraete FJM. Pediculectomy for thoracolumbar spinal decompression in the dachshund. J Am Anim Hosp Assoc 1994; 30: 233–238.
12. Hecht S, Thomas WB, Marioni-Henry K, et al. Myelography vs. computed tomography in the evaluation of acute thoracolumbar intervertebral disc extrusion in chondrodystrophic dogs. Vet Radiol Ultrasound 2009; 50: 353–359.
13. Garosi L. The neurological examination. In: Platt SR, Olby NJ, eds. BSAVA manual of canine and feline neurology. 3rd ed. Gloucester, England: British Small Animal Veterinary Association, 2004; 24–34.
14. Lorenz MD, Kornegay JN. Neurologic history and examination. In: Lorenz MD, Korneygay JN, eds. Handbook of veterinary neurology. 4th ed. St Louis: Elsevier Science, 2004; 45–74.
15. Bagley RS. Clinical examination of the animal with suspected neurologic disease. In: Bagley RS, ed. Fundamentals of veterinary clinical neurology. Ames, Iowa: Saunders, 2009; 57–108.
16. Fletcher TF, Kitchell RL. Anatomical studies on the spinal cord segments of the dog. Am J Vet Res 1966; 27: 1759–1767.
17. Fletcher TF, Kitchell RL. The lumbar, sacral and coccygeal tactile dermatomes of the dog. J Comp Neurol 1966; 128: 171–180.
18. Coates JR. Intervertebral disk disease. Vet Clin North Am Small Anim Pract 2000; 30: 77–110.
19. Garosi L. Lesion localization and differential diagnosis. In: Platt SR, Olby NJ, eds. BSAVA manual of canine and feline neurology. 3rd ed. Gloucester, England: British Small Animal Veterinary Association, 2004; 35–53.
20. deLahunta A, Glass G. Lower motor neuron: spinal nerve, general somatic efferent system. In: deLahunta A, Glass G, eds. Veterinary neuroanatomy and clinical neurology. 3rd ed. St Louis: Saunders, 2009; 77–133.
21. deLahunta A, Glass G. The neurologic examination. In: deLahunta A, Glass G, eds. Veterinary neuroanatomy and clinical neurology. 3rd ed. St Louis: Saunders, 2009; 487–501.
22. Van Buuren S, Groothuis-Oudshoorn K. Mice: multivariate imputation by chained equations in R. J Stat Softw 2011; 45: 1–67.
23. Hoeting JA, Madigan D, Raftery AE, et al. Baysian model averaging: a tutorial. Stat Sci 1999; 14: 382–417.
24. Viallefont V, Raftery AE, Richardson S. Variable selection and Bayesian model averaging in case-control studies. Stat Med 2001; 20: 3215–3230.
25. Sing T, Sander O, Beerenwinkel N, et al. ROCR: visualizing classifier performance in R. Bioinformatics 2005; 21: 3940–3941.
26. Robin X, Turck N, Hainard A, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 2011; 12:77.
27. Hosmer DW, Lemeshow S. Applied logistic regression. 2nd ed. Hoboken, NJ: John Wiley & Sons, 2000.
28. Zhou XH, Obuchowski NA, McClish DK. Statistical methods in diagnostic medicine. New York: John Wiley & Sons Inc, 2002.
29. Gutierrez-Quintana R, Edgar J, Wessmann A, et al. The cutaneous trunci reflex for localizing and grading thoracolumbar spinal injuries in dogs. J Small Anim Pract 2012; 53: 470–475.
30. deLahunta A, Glass G. Lower motor neuron: general visceral efferent system. In deLahunta A, Glass G, eds. Veterinary neuroanatomy and clinical neurology. 3rd ed. St Louis: Saunders, 2009; 168–191.
