Materials and Methods

Study design—Interrelationships among variables influencing interpretation and classification of herd infection status for JD were estimated by use of simulated data for various herd sizes, true within-herd prevalences, and sampling and testing methods. Four values of true prevalence (2%, 5%, 10%, and 15% perceived as very low, low, low to moderate, and moderate risk, respectively) were used to simplify model calculations. First, it was determined how these variables affected the number of test-positive cattle when a subset of cattle in a herd and an entire herd were tested for various simulated scenarios. Second, the confidence level of detecting infected herds was evaluated by estimating the probability of finding ≥ 1 infected animal in herds with true prevalences of 2%, 5%, 10%, and 15% for various testing methods and sample sizes. These estimates were used to construct figures (or tables) that could be used by veterinarians or herd owners to classify (with 95% confidence) herd infection status on the basis of the maximum number of test-positive cattle or culture-positive fecal pools detected.

Estimation of number of test-positive cattle on the basis of testing a subset of cattle in a herd and testing the entire herd—The number of test-positive cattle (≥ 36 months old) was estimated for various herd sizes (small herds, 50 and 100 cattle; medium herds, 200, 400, 600, and 800 cattle; and large herds, 1,000, 2000, 3,000, and 4,000 cattle) and true within-herd prevalences (2%, 5%, 10%, and 15%). A hypergeometric simple random sample of 30 cattle was obtained from all cattle in a herd by use of a built-in function in a commercially available software program^a on the basis of herd size, true number of MAP-infected cattle in the herd adjusted for imperfect sensitivity and specificity of testing method, number of samples per herd, and number of iterations. The sample size of 30 cattle was used because it is the minimum number of samples required for the entry level of the test-negative component of the VBJDCP. When testing of an entire herd was evaluated, the number of samples was equal to herd size. We used fixed values for sensitivity and specificity of 60% and 99.9% for IFC, 30% and 99% for ELISA, and 20% and 99.9% for ELISA-FC, respectively. The sensitivity and specificity were derived from a report⁹ of the consensus-based estimates. True prevalence was calculated by use of the following equation: (AP + [specificity − 1])/(sensitivity + [specificity − 1]), where AP is the apparent prevalence.^8,10 The sensitivity and specificity for PCR assay of fecal samples were assumed to be equivalent to those of IFC; hence, separate simulations were not performed for these tests. Then, 1,000 iterations for a sample size of 30 were performed, and the 2.5th, 50th (median), and 97.5th percentiles were evaluated to determine uncertainty in the number of test-positive cattle.

Estimation of probability of detecting ≥ 1 test-positive animal in a herd—For small, medium, and large herds, another built-in function in the commercially available software program^b was used to estimate the probability of finding test-positive cattle on the basis of the number of MAP-infected cattle in a herd, herd size, and number of samples tested per herd that was determined on the basis of hypergeometric random sampling. Within-herd true prevalence values were 2%, 5%, 10%, and 15%. Fixed values were used for the sensitivity and specificity of testing methods. The effect of an increase in the number of samples on the probability of detecting ≥ 1 test-positive animal was evaluated.

Classification of herd infection status on the basis of maximum number of test-positive cattle detected with 95% confidence—The number of test-positive cattle depends on within-herd prevalence, number of samples tested, testing method, and herd size. The maximum number of test-positive cattle detected for a given within-herd prevalence with a specified level of confidence was estimated by use of a built-in function for the inverse of the hypergeometric cumulative distribution function in a commercially available software program.^c The function calculated the maximum number of test-positive cattle that should be detected with 95% confidence in a sample of a specified number of cattle randomly selected from a herd of cattle, given that there were no more than a certain number of MAP-infected cattle in the herd. The estimate was performed on the basis of testing for the entire herd for herd sizes of 50 to 200 cattle, 4 values of within-herd prevalence (2%, 5%, 10%, and 15%), and 3 testing methods (IFC, ELISA, and ELISA-FC). The estimates were used to create a prediction line for the maximum number of test-positive cattle detected with 95% confidence for a specific value of within-herd prevalence.

Classification of herd infection status on the basis of the maximum number of culture-positive fecal pools detected with 95% confidence—A pool of 5 fecal samples from each of 5 randomly selected cattle was considered in the study because this pool size is commonly used in the United States. First, prevalence of infected fecal pools (ie, fecal pool in which there was ≥ 1 individual fecal sample from an infected animal) for herd sizes of 50 to 4,000 cattle and 4 values of true within-herd prevalence (2%, 5%, 10%, and 15%) were estimated. Prevalence of infected fecal pools was estimated by use of a built-in function in the commercially available software program^b on the basis of the number of infected cattle in a sample, number of JD-infected cattle in the herd, herd size, and sample size. When the number of infected cattle in a sample was 0 and sample size was 5, the function generated the probability that there were no samples from infected cattle in a pool size of 5 cattle. Hence, the prevalence of infected fecal pools in a herd was 1 – np, where n is the sample size and p is the probability of having no samples from infected cattle. The estimated prevalence of infected fecal pools was used to calculate the number of infected fecal pools adjusted for sensitivity and specificity of PFC testing, which were assumed to equal the values for IFC of 60% and 99%, respectively.

Next, the maximum number of fecal pools with positive culture results (ie, culture-positive fecal pools) detected with 95% confidence by testing all cattle in a herd by use of PFC was estimated. This was accomplished by use of a built-in function in the commercially available software program^c as previously described on the basis of the total number of fecal pools in a herd, number of infected fecal pools (calculated as the total number of pools minus the number of pools that had 0 infected cattle), and number of tested fecal pools. Outcomes from the simulations were used to create a prediction line for the maximum number of culture-positive fecal pools detected with 95% confidence for a specific value of within-herd prevalence.

Results

Estimation of the number of test-positive cattle on the basis of testing a subset of cattle in a herd and testing the entire herd—Testing a subset of cattle in a herd revealed that there were 0 test-positive cattle detected in some iterations of 30 samples for all categories of within-herd prevalence (2%, 5%, 10%, and 15%) and herd sizes (50 to 4,000 cattle) for all types of testing methods (Table 1). When IFC was the test used, 0 to 10 test-positive animals were detected, depending mainly on within-herd prevalence. Herd size had minimal or no effect on the number of test-positive cattle detected; however, there was a larger sampling variation for small herds versus large herds. Overall, infected herds with low within-herd prevalence were likely to be misclassified when testing 30 cattle/herd regardless of the type of testing method used. Infection was not detected in some moderate-prevalence (eg, 15%) herds, and the chance of not detecting infected cattle was greater when the ELISA-FC was used.

Results of the 3 testing methods when an entire herd was tested were summarized (Table 2). There was no sampling variation when the entire herd was tested; therefore, the estimate was a single number. Most infected herds were detected by use of whole-herd testing, except when the ELISA-FC was used in small herds (≤ 100 cattle) with low prevalence (< 5%). In addition to the cost, the main limitation of whole-herd testing was misclassification of noninfected herds. All antibody and organism-detection tests yielded some false-positive results when the prevalence was 0%. When an entire herd was tested (ie, whole-herd testing) with the ELISA (specificity, 99%), ≥ 1 false-positive animal was found for all herd sizes. Even when test specificity was as high as 99.9%, as in the case of the IFC and ELISA-FC, false-positive results were evident when herd size was ≥ 500 cattle.

Estimation of the probability of detecting ≥ 1 test-positive animal in a herd—Results of scenarios in which various proportions of a herd were tested by use of IFC, ELISA, and ELISA-FC for herd sizes between 50 and 4,000 cattle and true within-herd prevalence of 5% were summarized (Table 3). Overall, the probability of detecting ≥ 1 test-positive animal increased as sample size increased. The proportion of a herd required to be tested to achieve results with 95% confidence increased as within-herd prevalence decreased, and the proportion changed as a function of within-herd prevalence.

When testing was conducted by use of the ELISA, estimates had a pattern (a convex slope) similar to that obtained when the IFC was used for testing, and the proportion of herd required to be tested to achieve 95% confidence was slightly higher than the proportion when testing was performed with the IFC. Results determined on the basis of testing with the ELISA-FC indicated that this testing method required a substantially larger proportion of the herd to be tested than when the IFC and ELISA were the testing methods. Whole-herd testing by use of the ELISA-FC did not achieve 95% confidence of having ≤ 1% prevalence when herd size was < 200 cows (data not shown).

Classification of herd infection status on the basis of the maximum number of test-positive cattle detected with 95% confidence—The maximum number of test-positive cattle that were detected with 95% confidence when whole-herd testing was conducted via IFC, ELISA, and ELISA-FC was estimated. A prediction line was created for each level of risk. For example, if all cattle in a herd of 100 cattle were tested via IFC, the maximum number of test-positive cattle should not exceed 9, 6, 3, and 1 for classifying the herd as having a true prevalence of ≤ 15%, ≤ 10%, ≤ 5%, and ≤ 2%, respectively. Alternatively, if 4 cattle were found to be test-positive by use of the IFC, the herd could be classified as having a maximum true prevalence of between 5% and 10% (Figure 1). For whole-herd testing by use of the ELISA, estimates of the maximum number of test-positive cattle were smaller than the estimates obtained by use of the IFC for the same classification levels (Figure 2). Whole-herd testing with the ELISA-FC (Figure 3) was not adequate for classifying infection status when herds were small or had low prevalence (< 5%), which was consistent with the findings for use of the ELISA-FC with regard to the probability of detecting ≥ 1 infected animal (Table 2). When the maximum number of test-positive cattle detected was 0, it was necessary to determine the probability of detecting ≥ 1 infected animal by use of the testing method to decide whether the value of 0 was attributable to a lack of MAP infection in the herd or a lack of power to detect infected cattle.

View Full Size

Graph of the maximum number of cattle with positive test results (test-positive cattle) for 4 values of true within-herd prevalence of MAP determined on the basis of testing of the entire herd (ie, whole-herd testing) by use of IFC. Levels 1 (solid black line), 2 (dashed black line), 3 (dotted black line), and 4 (dotted-and-dashed black line) were based on 95% confidence that the true within-herd prevalence was ≤ 15%, ≤ 10%, ≤ 5%, and ≤ 2%, respectively.

Citation: American Journal of Veterinary Research 73, 2; 10.2460/ajvr.73.2.248

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Graph of the maximum number of test-positive cattle for 4 values of true within-herd prevalence of MAP determined on the basis of whole-herd testing by use of ELISA. See Figure 1 for remainder of key.

Citation: American Journal of Veterinary Research 73, 2; 10.2460/ajvr.73.2.248

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Graph of the maximum number of test-positive cattle for 4 values of true within-herd prevalence of MAP determined on the basis of whole-herd testing by use of ELISA-FC. See Figure 1 for remainder of key.

Citation: American Journal of Veterinary Research 73, 2; 10.2460/ajvr.73.2.248

• Download Figure
• Download figure as PowerPoint slide

Classification of herd infection status on the basis of the maximum number of culture-positive fecal pools detected with 95% confidence—The maximum number of culture-positive fecal pools that could be detected with 95% confidence when fecal samples from an entire herd were placed into pools of 5 individual fecal samples/pool and tested by culture (ie, PFC) was estimated (Figure 4). For example, in a 100-cow herd in which 20 fecal pools were created, detection of 6, 4, 2, and 1 culture-positive fecal pools indicated a true within-herd prevalence of ≤ 15%, ≤ 10%, ≤ 5%, and ≤ 2%, respectively.

View Full Size

Graph of the maximum number of pooled fecal samples with positive culture results (test-positive fecal pools) for 4 values of true within-herd prevalence of MAP determined on the basis of whole-herd testing of pooled fecal samples. Each pool contains individual fecal samples from 5 cattle. See Figure 1 for remainder of key.

Citation: American Journal of Veterinary Research 73, 2; 10.2460/ajvr.73.2.248

• Download Figure
• Download figure as PowerPoint slide

Discussion

In the study reported here, the effect of herd size, true within-herd prevalence, and sampling and testing methods on interpretation and classification of herd JD infection status was evaluated via simulated data. Then, a new system was created for classifying JD infection status of US cattle herds on the basis of the probability of detection of infection and the maximum number of test-positive cattle detected. Categorization of herd JD infection status into infected versus noninfected herds often requires testing many cattle and high cost, and most of the time, categorization is not feasible when tests are imperfect, within-herd prevalence is very low (eg, < 2%), and additional evidence for classification of herd status is not available. It is extremely difficult and financially not feasible to differentiate a noninfected population from an infected population with a very low prevalence. Increasing the number of cattle tested in each herd to increase herd sensitivity often affects herd specificity if the herd is not infected and the testing method used is < 100% specific. Most available testing methods for paratuberculosis are highly specific, but all have specificities < 100%, which could result in misclassification of noninfected herds, depending on the number of cattle tested.

For JD control purposes, it is important to assess herd infection prevalence to a meaningful level (eg, < 5% and 10%) in terms of risk of disease transmission. Such information will facilitate improved planning and monitoring of control programs that are financially feasible and provide a proactive warning to cattle producers regarding the risk of MAP infection.

Classification of herd infection status in the present study was performed by the use of all information available, including the proportion of test-positive samples detected and the probability of detection given the sampling and testing methods used. Herds were classified into categories that represented the risk of JD transmission instead of being classified as test-negative or test-positive herds. Classification on the basis of risk of JD transmission provides reasonable classification of herd infection status to the extent that a participating owner could afford or would be willing to have their herd classified without having to prove its MAP-free status. The concept and new system for classifying JD status of US cattle herds developed in the present study may also be applicable to risk categorization for other infectious diseases, provided there are estimates of test sensitivity and specificity.

For the present study, herd JD infection status was categorized on the basis of the maximum true within-herd prevalence. True prevalence was used as the benchmark because the measure allows comparison of results of various tests after adjustment for test sensitivity and specificity. The sensitivity and specificity of tests eligible for use in herd tests were determined on the basis of consensus-based estimates.⁹ To better evaluate the variability of results attributable to other factors, the uncertainty in test sensitivity and specificity was not included in the model. However, if effects of these other factors were included, a small uncertainty and therefore a minor effect on the herd infection classification would be anticipated. Four categories of risk were defined as levels 1, 2, 3, and 4. Level 1 was based on 95% confidence that the true within-herd prevalence was ≤ 15%. For example, 15% true prevalence is equal to a test prevalence of 5.3%, assuming testing of serum or milk samples was conducted by use of an ELISA with 30% sensitivity and 99% specificity Levels 2, 3, and 4 were based on 95% confidence that the true within-herd prevalence was ≤ 10%, ≤ 5%, and ≤ 2%, respectively These categories indicated high confidence of an increasingly lower risk of infection with JD but provided no direct indication of the probability the herd was not infected, which would necessitate Bayesian inference. Overall, this method of classification provides a transparent system to categorize participating herds on the basis of maximum within-herd prevalence and herd testing methods.

The use of PFC as an alternative cost-saving testing method for JD herd classification was evaluated. The PFC method was based on a maximum of 5 fecal samples/pool, which is commonly used in the United States. Simulation results indicated that PFC was capable of classifying herd JD infection status as a true within-herd prevalence of ≤ 5% and ≤ 2% in herds of 40 and 90 cows, respectively. In addition, it is possible to test feces from individual cows contributing samples to test-positive fecal pools, calculate the percentage of culture-positive cows, and then assign a classification level by use of the thresholds for IFC. For this study, we assumed the sensitivity (60%) and specificity (99%) of PFC were the same as those of IFC, which might have overestimated the ability of PFC for MAP detection if infected cattle were low shedders of MAP in feces.

We anticipate that results of the present study will be useful for USDA APHIS Veterinary Services policymakers to achieve more efficient control of JD in US cattle populations through modification of the current test-negative program by use of concepts generated from this study. Key concepts include herd classification levels for JD based on critical threshold values for herd testing on the basis of herd size. Some of the classification levels allow for positive test results (through false-positive or true-positive test results). A new classification system for JD status in cattle herds could provide more low-cost and scientifically sound alternatives for herds to achieve lower risk than are provided by the current test-negative program. Under a new system, lower-cost testing methods such as ELISA or PCF could potentially be used at any of the risk levels, and use of the most costly testing strategy (IFC) would be minimized. At the same time, a new system would retain a high likelihood of very low risk in the lowest risk category (level 4). This lowest risk category could also be modified or expanded to include additional levels with 0 test-positive cattle. Under the current program, use of ELISA or PFC is restricted at some of the lower risk levels, which potentially limits producer incentives for use of the program to make progress from one level to another.

Further evaluation of another low-cost testing strategy, environmental testing, is needed, especially in dairy and beef cattle herds with a low prevalence of MAP infection. This should include the effect of environmental sampling of cattle herds on the basis of group or production string. We did not include the evaluation of environmental fecal samples in the present study because of a lack of available data.

Establishment of countries, regions, and zones as being free from animal pathogens is an important issue for animal trade. Investigators have evaluated the methods for calculating the exact probability of detecting infected animals when considering both imperfect tests and finite population size,¹¹ sample size requirement for surveys designed to provide evidence of freedom from disease,¹² and surveillance as a tool to establish freedom from disease by use of multiple data sources.¹³ Classification of a population as infection-free is difficult, costly, and time-consuming, especially when differentiating a noninfected population from a population with a very low prevalence. Recently, use of a risk-based approach has been proposed as a sampling alternative to increase confidence of detecting pathogens in very low–prevalence populations¹⁴ and to minimize the required sample size in a population with a very low prevalence.¹⁵ However, this requires knowledge of heterogeneous risk among subgroups in a population. In the present study, we focused on infected herds with low and very low prevalence to avoid confusion arising from differentiating them from noninfected herds. In addition, we have provided farmers and veterinarians with a reference that can be used for classification of JD infection status in cattle.

On the basis of the findings from the present study, there are several advantages to an updated herd classification program. One is more transparent interpretation of herd JD classification by removal of the concept of freedom from JD. Instead, we propose the concept of a 95% confidence of a true within-herd prevalence less than or equal to a defined value, which could be used as a standard to which future tests are compared. Interpretation of test results by herd size provides a testing scheme that is more equitable across herds of all sizes. A major outcome of this classification program is lower-cost alternatives for testing (use of ELISA alone and testing of pooled fecal samples) in specific classification levels. This program could facilitate increased participation in the voluntary classification program through removal of the need to confirm test-positive results for the ELISA and faster progression to levels 3 and 4 with an associated lower risk of infection. Finally, this system would maintain a high degree of rigor throughout the classification program to ensure scientific credibility of the program. Therefore, results from the present study indicate that cattle herds can be classified in a scientifically credible manner on the basis of risk of JD infection status by use of available diagnostic tests. In fact, based in part on findings from the study reported here, USDA APHIS Veterinary Services published new Uniform Program Standards for the VBJDCP in September 2010. Furthermore, as of October 20, 2011, 15 states have adopted these new program standards.