Orthopedic diseases are common in dogs, and visual evaluation of gait is often used for localizing and diagnosing musculoskeletal disease.1 A more objective method to diagnose abnormal gait in dogs is IGA, with the most commonly used equipment being force plates and pressure-sensing walkway systems.2–4 However, it is difficult to compare IGA data obtained in separate laboratories because of a lack of standardization of data acquisition methods and because IGA procedures are not always described in detail in published reports. For example, the definition of a valid trial varies among laboratories,5–7 and descriptions of equilibration and calibration of the pressure-sensing walkway are not always reported.8,9
In addition to variations in IGA data collection and interpretation, lack of statistical power has also been an issue. This is exemplified in a recent study10 that evaluated randomized controlled clinical trials involving small animals in which only 14 of the 103 (14%) trials with negative results were sufficiently powered to detect a 25% relative difference in outcome between treatments. To improve the power (ie, increase sample size) of IGA research, collaborative studies with gait data obtained in separate gait laboratories may be needed. Although a study11 shows that cooperative work on IGA has occurred, differences in gait analysis systems and brands and data acquisition protocols largely prevent such collaborative research, unless the transferability of IGA data between laboratories is first established. To our knowledge, methods for animal gait laboratories to validate the use of IGA data from other animal gait laboratories are not available.
Historically, most IGA research in animals has been performed by comparing results of IGA between groups (eg, experimental vs control) of dogs when evaluating outcomes following particular treatments.12,13 However, in a clinical setting, the interest is in an individual dog. Two approaches have been used to discern clinically normal versus abnormal gaits in individual dogs. With 1 approach, IGA results for dogs with known lameness (ie, cases) versus results for dogs without lameness (ie, noncases or controls) are evaluated with ROC curve analysis.14 For each gait variable assessed with ROC curve analysis, a single statistically optimal cutoff value and its sensitivity and specificity for detection of gait abnormalities can be determined. With another approach, IGA results from clinically normal dogs free of orthopedic and neurologic disease are used to establish clinically normal values or RIs, each with a lower and upper reference limit and a corresponding specificity, usually set at a 95% CI. Both ROC curve analysis and development of RIs need sufficient sample size and sample makeup that are representative of the population of interest. Because case animals are included in ROC curve analysis, such derived sensitivity can be directly generalized only to the disease or diseases represented in the case animals. This restriction does not apply to RIs because, in their development, only noncase animals are included; however, RI sensitivity can only be determined post hoc. In the PU-AGL, RIs for kinetic variables in walking dogs were recently established15,a with methods recommended by the CLSI and comparable to those recommended by the CLSI for clinical pathology laboratories that define, establish, and verify clinicopathologic RIs.16–18
Establishing RIs for laboratory variables can be time-consuming and costly; therefore, it is not always feasible for each laboratory to develop its own RIs. An alternative is to adopt RIs established in another laboratory, and from our experience, this approach is particularly attractive in veterinary research that is often constrained by small sample sizes. However, for such RIs to be valid for the adopting laboratory, a transference process needs to be followed. To validate the transference of RIs, the CLSI recommends 3 approaches: subjective assessment for comparability of the analytic system and the test subject population, evaluation of a large number of reference individuals, or statistical assessment of a small number of reference individuals (ie, the small data approach).16,18,19 The objective of the study reported here was to use the small data approach of the CLSI as a method to evaluate the transferability of RIs for kinetic variables of IGA in walking dogs from an RI-originator laboratory (the PU-AGL) to another laboratory (the CSU-AGL) that used the same data acquisition and analytic techniques for IGA in dogs. We hypothesized that the RIs developed at the PU-AGL would be transferable to the CSU-AGL.
Materials and Methods
Model
The present validation study was designed and analyzed according to the recommendations of the CLSI for use of the small data approach for validating transference of RIs.16,18,19 Data collected from dogs that underwent IGA at the CSU-AGL were compared with historical IGA data and previously established RIs from the PU-AGL.15,a The study protocol was approved by the institutional animal care and use committees at both universities.
Animals
Healthy client-owned dogs were enrolled in the study for IGA at the CSU-AGL, and solicitation for participation was performed by announcement of a clinical trial to the public. Owner consent was obtained for each dog. Inclusion criteria were that dogs had to be skeletally mature and orthopedically healthy on the basis of a history of orthopedic soundness and results of a physical examination performed by a board-certified small animal surgeon. To ensure that the CSU-AGL study population resembled the PU-AGL reference population, dogs were assigned to 1 of 4 groups on the basis of body weight (10 to < 20 kg, 20 to < 30 kg, 30 to < 39 kg, or 39 to < 70 kg), with an intended 5 dogs/group. To further facilitate a heterogeneous study population, dogs were of different breeds, when possible.2 Instrumented gait analysis of additional dogs was performed to serve as replacements for outliers detected during the RI transference validation procedure.
CSU-AGL gait analysis
The CSU-AGL used the same pressure-sensing walkway systemb covered by the same protective matc as did the PU-AGL for data collection.20 The CSU-AGL also calibrated and equilibrated the walkway prior to the study, and calibration files for the walkway were created by use of a phantom dog (a 3-legged stool with weights of various amounts added according to the manufacturer guidelines) from the PU-AGL and manufacturer's information for systemb settings at 40 PSI, 75 PSI, and 125 PSI. Calibration files were created and saved at 39.9 kg/40 PSI, 58.6 kg/75 PSI, and 70.3 kg/125 PSI for small-, medium-, and large-sized dogs, respectively. Before the start of IGA data collection for each new dog, the equilibration file (on the basis of the dog's size) was entered and then the calibration file (also on the basis of the dog's size) was loaded. Small-sized dogs were defined as those weighing 10 to < 20 kg. Medium-sized dogs were defined as those weighing 20 to < 39 kg. Large-sized dogs were defined as those weighing 39 to < 70 kg.
For each dog, IGA data collection was performed during a single visit at the CSU-AGL, and each trial was video recorded.d Each dog was walked at its preferred velocity over a pressure-sensing walkway in accordance with the PU-AGL protocol. The same handler walked all of the dogs for their IGA, and the gait laboratory technician from the PU-AGL was present in the CSU-AGL for all data collection. At least 10 trials were recorded for each dog, and data from 6 valid trials were used for analysis. For each IGA variable, the mean value resulting from the 6 trials was considered the variable value for that individual dog and was used for the transference procedure. A trial was considered valid if the dog walked straight forward without stopping, hesitating, trotting, pacing, pulling on the lead, or having overt head movement.5 Following completion of data collection, the PU-AGL technician analyzed and compiled the IGA data for evaluation of PVF (N), PVF-CV, PVF-SI, DWD (relative body weight distribution21), DWD-CV, and DWD-SI for each limb of each dog. The CVs (interstep variabilities) and SIs were calculated as previously described.5
PU-AGL RIs
The PU-AGL had previously established RIs for IGA variables in orthopedically and neurologically normal dogs (n = 72).15,a These RIs were developed on the basis of the 95% CIs for gait variables evaluated with the robust method as described in the CLSI guidelines.16–18 On the basis of that previous work,15,a we used the same grouping of dogs by body weight, equilibration and calibration of equipment, and criteria for valid walking trials for the CSU-AGL.
Statistical analysis
Transferability of IGA RIs from the PU-AGL to the CSU-AGL was validated with the small data approach, according to the CLSI recommendations.16 Briefly, the Tukey test was used to detect outliers in the IGA results for the initial 20 dogs grouped by body weight and tested at the CSU-AGL. Each such outlier from an individual dog was then replaced with the measurement obtained of a dog randomly selected from the remaining 7 dogs enrolled in the study. This procedure was repeated until no outlier result for an individual dog was detected. There were 20 variables that included data sets with results from 20 dogs. The results were considered to follow a binomial distribution (each dog's IGA variable was either within or outside of the respective RI); thus, the PU-AGL-developed RI for a particular variable was deemed transferable to the CSU-AGL if ≤ 2 dogs represented in the CSU-AGL data set had results outside of the RI for that variable. If, on initial review, 3 or 4 dogs represented in the data set had results outside of the PU-AGL-developed RI for that particular variable, then another group of 20 dogs would be recruited to create a new data set at the CSU-AGL, and the analysis would be repeated. In the second analysis, if > 2 dogs represented in the data set had results outside of the RI for that variable, then that particular RI was considered not transferable in accordance with the CLSI recommendations.16 Similarly, on initial review, if ≥ 5 dogs represented in the data set had results outside of the RI for a particular variable, then that particular RI was considered not transferable. Type I error rate (rejection of the PU-AGL-developed RI when it was valid) was set at P < 0.01.
A linear model was used to establish weight-dependent RIs for PVF to compare results for the dogs tested at the CSU-AGL with historical results15,a for dogs tested at the PU-AGL. In analysis of regression coefficients, values of P ≤ 0.05 were considered significant. Statistical analysis was performed with available software.e
Results
Animals
Twenty-seven client-owned dogs were enrolled in the study: mixed-breed dogs (n = 7), Border Collies (3), Golden Retrievers (2), Labrador Retrievers (2), and 1 each of a Beagle, Boxer, German Shepherd Dog, Great Dane, Great Pyrenees, and Portuguese Water Dog. There was 1 sexually intact male, 11 castrated males, and 8 spayed females. At IGA, the mean ± SD weight was 31.0 ± 13.4 kg (range, 11.0 to 66.8 kg) and the mean ± SD age was 5.4 ± 3.4 years (range, 1 to 15 years). Twenty of these 27 dogs were assigned to 1 of 4 groups on the basis of body weight (10 to < 20 kg, 20 to < 30 kg, 30 to < 39 kg, and 39 to < 70 kg), with 5 dogs in each group. Data from the remaining 7 dogs were used for replacement measurements of gait variables when outliers were identified among results for the initial 20 dogs.
Transference of RIs
By use of the Tukey test, 9 results for individual dogs were identified as outliers across 6 variables, and each of these outliers was replaced with a value from a dog randomly selected from the remaining 7 dogs. When CSU-AGL results were compared with the RIs from the PU-AGL, ≤ 2 dogs represented had values outside of the RIs for DWD, DWD-SI, PVF-CV, and PVF-SI (Table 1); therefore, the RIs for these variables met the criteria for transferability to the CSU-AGL. In addition, 3 dogs had a DWD-CV of the right forelimb < 0.02 (the lower reference limit of the PU-AGL-derived RI), and because the clinical importance of a DWD-CV below the lower reference limit was unknown, the RI of DWD-CV of the right forelimb was also accepted as transferable. However, 15 of the 20 dogs tested at the CSU-AGL had PVF values that were outside of the RIs for their respective body weight groups; thus, the PU-AGL-derived, weight-specific RIs for PVF were not transferable to the CSU-AGL.
Results of analysis with the small data approach of the CLSI to determine transferability of RIs for kinetic variables (DWD, DWD-CV, DWD-SI, PVF, PVF-CV, PVF-SI) of IGA in dogs from the laboratory that developed the RIs (the PU-AGL) to another laboratory that used the same data acquisition and analytic techniques for IGA in walking dogs (CSU-AGL).
Variable | Limb | No. of dogs with mean results outside of the RI* (n = 20) | RI* |
---|---|---|---|
DWD | Left forelimb† | 2 | 26.3–32.9 |
Right forelimb† | 0 | 25.2–32.5 | |
Left hind limb† | 1 | 17.1–23.9 | |
Right hind limb† | 1 | 17.3–24.6 | |
DWD-CV | Left forelimb† | 1 | 0.02–0.08 |
Right forelimb† | 2 | 0.02–0.09 | |
Left hind limb† | 1 | 0.03–0.12 | |
Right hind limb† | 0 | 0.02–0.10 | |
DWD-SI | Forelimbs† | 2 | 0.19–8.17 |
Hind limbs† | 0 | 0.12–14.13 | |
PVF (N) | Left forelimb | 18 | ([0.41 × BW] – 0.95)–([0.4l × BW] + 4.84) |
Right forelimb | 18 | ([0.40 × BW] – 0.89)–([0.40 × BW] + 5.11) | |
Left hind limb | 12 | ([0.20 × BW] – 0.60)–([0.30 × BW] + 3.80) | |
Right hind limb | 14 | ([0.21 × BW] – 0.58)–([0.31 × BW] + 3.18) | |
PVF-CV | Left forelimb† | 1 | 0.02–0.11 |
Right forelimb† | 2 | 0.02–0.09 | |
Left hind limb† | 0 | 0.02–0.13 | |
Right hind limb† | 2 | 0.03–0.12 | |
PVF-SI | Left forelimb† | 2 | 0.15–8.13 |
Right forelimb† | 0 | 0.13–14.06 |
BW = Body weight in Newtons.
RI established at the PU-AGL.l5a
RI transferable from the PU-AGL to the CSU-AGL.
To further evaluate the nontransferability of PVF RIs, scatterplots of PVF versus body weight were as-sessed for results obtained at the CSU-AGL versus the PU-AGL. The regression slopes were significantly (P < 0.05) greater for the left and right forelimbs and the left and right hind limbs of dogs tested at the CSU-AGL (0.6105, 0.6127, 0.3291, and 0.3969, respectively), compared with historical results for dogs tested at the PU-AGL (0.4081, 0.3996, 0.2522, and 0.2603, respectively; Figures 1 and 2).
Discussion
Results of the present study indicated that use of the small data approach of the CLSI as a method to validate the transferability of selected PU-AGL-derived RIs for evaluating orthopedically sound dogs at the CSU-AGL was simple, inexpensive, and effective. Our findings indicated that RIs for DWD, DWD-SI, DWD-CV, PVF-SI, and PVF-CV were transferable from the PU-AGL to the CSU-AGL, whereas the weight-specific RIs for PVF were not.
The transferability validation technique used in the present study is simple and can be performed with 20 orthopedically sound dogs as representatives of the general healthy dog population and does not require complicated statistical analysis. Because the RIs of interest were based on 95% CIs, the probability for a result from a dog tested at the CSU-AGL to have been outside of the respective RI followed a binomial distribution. This simple quantitative analysis technique, compared with a subjective assessment for transference, proved valuable because, although the PU-AGL and CSU-AGL analytic system and test subject population were comparable according to the CLSI guidelines for subjective assessment for transference,16 suggesting RI transferability, the simple quantitative technique we used identified the weight-specific RIs for PVF as not transferable. In addition, although this quantitative technique originally was described for the validation of transference, it may also be used to periodically validate RIs in the laboratory that established the RIs and to detect drift caused by aging of equipment over time.
The RIs that were transferable in the present study were the dimensionless ratios of DWD, DWD-SI, DWD-CV, PVF-SI, and PVF-CV that had numerators and denominators with the same dimensions and, therefore, were descriptors independent of body characteristics, such as body weight, body shape, and limb length. Our findings that the weight-specific RIs for PVF were not transferable suggested that this body weight function differed between the PU-AGL and CSU-AGL. This difference was most likely related to data acquisition. More specifically, because we made great efforts to duplicate the PU-AGL data acquisition method and studya population, we attributed the difference to variation between the 2 pressure-sensing walkways. However, without transporting the exact walkway equipment from the PU-AGL to test both walkways at the CSU-AGL, it was not possible to completely rule out the possibility of variation in calibration between the walkways. In addition, the walkway used at the PU-AGL was older than the one at the CSU-AGL, and differences in age of the pressure sensors could have resulted in variability of results. To our knowledge, there are no published studies on how the age of a pressure-sensing walkway could impact data collection or how the accuracy of pressure sensors may change over the lifespan of a walkway.
Our approach in the present study was conservative. We used the same gait analysis system from the same manufacturer, same acquisition protocols, and same technician as did the PU-AGL, which established RIs we evaluated for transference. With this study design, in combination with the detected calibration differences between the PU-AGL and CSU-AGL pressure-sensing walkways, our findings suggested that the transferability of dimensionless ratios in the present study may have been only minimally impacted by the calibration of the walkway. By extension, one could argue that these ratios may not be highly impacted by the brand or type of acquisition equipment (force platform or walkway). This should be further explored because transferability of dimensionless ratio RIs, regardless of the type of IGA equipment, would greatly enhance opportunities for collaboration between laboratories.
Because the IGA acquisition protocol and technician were the same at the 2 institutions, the effect of acquisition protocol on transferability could not be determined. However, intuitively, the acquisition protocol may be a substantial determinant of the transferability of RIs. This would suggest the potential value of a standardized gait acquisition protocol. The use of a standardized protocol may enhance the quality of IGA studies with more uniform collection of data and help to facilitate clinical and research collaborations between gait laboratories. At present, calibration and acquisition protocols at separate laboratories may vary or may not be described at all. Standard methods for calibration of force platforms have been published22; however, standard methods for calibration of pressure-sensing walkways are not available. Furthermore, design criteria for acquisition protocols in dogs do not exist, and there is no evidence that one protocol may be better than others. Thus, there is a need to develop design criteria and standardized acquisition protocols for IGA in dogs.
The primary limitations of the present study were attributable to our tight control of the variability of the analytic system and the test subject population. In the present study, the CSU-AGL test population mimicked that used by PU-AGL,15,a which reflected dog breeds in Indiana but not necessarily in Colorado; thus, our experimental design may have biased the outcome of the validation in favor of transference. In addition, we attempted to control the analytic system at the CSU-AGL by using the same model of the pressure-sensing walkway with the same protective cover, same calibration and equilibration of the walkway, and same AGL technician as used at the PU-AGL. To determine the true transferability of RIs from the PU-AGL to the CSU-AGL, additional studies should be performed at the CSU-AGL without the PU-AGL technician present.
The importance of the present study was that it demonstrated a simple test for validation of transference of RIs for IGA variables that could facilitate clinical and research collaborations between gait laboratories. The test could also be used to periodically validate RIs in the laboratory that established them and to detect drift caused by aging of equipment over time. Finally, the availability of a method to validate transference of RIs for IGA variables also reemphasizes the need for standardization of data acquisition techniques and more rigorous guidelines for calibration and equilibration of equipment used for IGA in dogs.
ABBREVIATIONS
CLSI | Clinical and Laboratory Standards Institute |
CSU-AGL | Colorado State University Animal Gait Laboratory |
CV | Coefficient of variation |
DWD | Dynamic weight distribution |
IGA | Instrumented gait analysis |
PSI | Pounds per square inch |
PU-AGL | Purdue University Animal Gait Laboratory |
PVF | Peak vertical force |
RI | Reference interval |
ROC | Receiver operating characteristic |
SI | Symmetry index |
Footnotes
Alshehabat MA. Instrumented gait analysis to characterize pelvic limb ataxia in dogs. PhD thesis, Department of Veterinary Clinical Sciences, College of Veterinary Medicine, Purdue University, West Lafayette, Indiana, 2012.
HRV Walkway 6 VersaTek System, Tekscan Inc, South Boston, Mass.
Smooth Top Easy Liner (20 inches × 6 feet), ShurTech Brands LLC, Avon, Ohio.
EQ900F, EverFocus Co Ltd, New Taipei City, Taiwan.
SPSS Statistics, version 24, IBM Corp, Armonk, NY.
References
1. Carr BJ, Dycus DL. Canine gait analysis. Today's Vet Pract 2016;93–100.
2. Besancon M, Conzemius M, Derrick T, et al. Comparison of vertical forces in normal greyhounds between force platform and pressure walkway measurement systems. Vet Comp Orthop Traumatol 2003;16:153–157.
3. Lascelles BD, Roe SC, Smith E, et al. Evaluation of a pressure walkway system for measurement of vertical limb forces in clinically normal dogs. Am J Vet Res 2006;67:277–282.
4. Budsberg SC, Verstraete MC, Soutas-Little RW. Force plate analysis of the walking gait in healthy dogs. Am J Vet Res 1987;48:915–918.
5. Kim J, Kazmierczak KA, Breur GJ. Comparison of temporospatial and kinetic variables of walking in small and large dogs on a pressure-sensing walkway. Am J Vet Res 2011;72:1171–1177.
6. Agostinho FS, Rahal SC, Geraldo B, et al. Influence of calibration protocols for a pressure-sensing walkway on kinetic and temporospatial parameters. Vet Comp Orthop Traumatol 2015;28:25–29.
7. Hans EC, Zwarthoed B, Seliski J, et al. Variance associated with subject velocity and trial repetition during force platform gait analysis in a heterogeneous population of clinically normal dogs. Vet J 2014;202:498–502.
8. Keebaugh AE, Redman-Bentley D, Griffon DJ. Influence of leash side and handlers on pressure mat analysis of gait characteristics in small-breed dogs. J Am Vet Med Assoc 2015;246:1215–1221.
9. Gordon-Evans WJ, Evans RB, Conzemius MG. Accuracy of spatiotemporal variables in gait analysis of neurologic dogs. J Neurotrauma 2009;26:1055–1060.
10. Giuffrida MA. Type II error and statistical power in reports of small animal clinical trials. J Am Vet Med Assoc 2014;244:1075–1080.
11. Arya C, Sharma S. Authorship trends and collaborative research in veterinary sciences: a bibliometric study. Chin Librarianship 2011;34:38–47.
12. Budsberg SC, Johnston SA, Schwarz PD, et al. Efficacy of etodolac for the treatment of osteoarthritis of the hip joints in dogs. J Am Vet Med Assoc 1999;214:206–210.
13. Vilar JM, Morales M, Santana A, et al. Controlled, blinded force platform analysis of the effect of intraarticular injection of autologous adipose-derived mesenchymal stem cells associated to PRGF-Endoret in osteoarthritic dogs. BMC Vet Res 2013;9:131.
14. Evans R, Horstman C, Conzemius M. Accuracy and optimization of force platform gait analysis in Labradors with cranial cruciate disease evaluated at a walking gait. Vet Surg 2005;34:445–449.
15. Breur G, Alshehabat M, Kazmierczak K, et al. Reference intervals for kinetic variables in walking dogs with different body weight and body shape. Vet Surg 2015;44:E46.
16. Clinical and Laboratory Standards Institute. Defining, establishing, and verifying reference intervals in the clinical laboratory; approved guideline. 3rd ed. Wayne, Pa: Clinical and Laboratory Standards Institute, 2008.
17. Geffré A, Braun J-P, Trumel C, et al. Estimation of reference intervals from small samples: an example using canine plasma creatinine. Vet Clin Pathol 2009;38:477–484.
18. Geffré A, Friedrichs K, Harr K, et al. Reference values: a review. Vet Clin Pathol 2009;38:288–298.
19. Tate JR, Yen T, Jones GR. Transference and validation of reference intervals. Clin Chem 2015;61:1012–1015.
20. Kieves NR, Hart JL, Evans RB, et al. Comparison of three walkway cover types for use during objective canine gait analysis with a pressure-sensitive walkway. Am J Vet Res 2019;80:265–269.
21. Kano WT, Rahal SC, Agostinho FS, et al. Kinetic and temporospatial gait parameters in a heterogeneous group of dogs. BMC Vet Res 2016;12:2.
22. ASTM International. Standard test method for verification of multi-axis force measuring platforms. ASTM Standard F3109–16. West Conshohocken, Pa: ASTM International, 2016.