Introduction
Scientific communication should be clear, succinct, and, as far as possible, unambiguous. Numerous medical and a few veterinary studies since the 1980s have examined the potential miscommunication between pathologists (clinical and anatomic) writing reports about submitted samples and clinicians who obtain, interpret, and ultimately act on the information contained within these reports.1–11 Most studies have focused on qualitative terms used by pathologists that denote a level of confidence associated with an interpretation of the submitted sample. These studies have demonstrated disconnect between the intended degree of certainty (or uncertainty) that the pathologists hope to convey with specific modifying terms and the interpretation of these terms by clinicians. Such disconnect, if substantial, could result in inappropriate clinical decisions, leading to patient harm.
Several investigators have proposed alternative approaches to describing the degree of certainty ascribed to a diagnosis by a pathologist.1–3,8,12,13 These include the use of a numeric scale (eg, assigning a percentage range), use of a standardized set of modifying terms that are clearly defined and nonoverlapping, or reducing the degree of certainty to a very limited list of options. Most studies have found variable enthusiasm for such schemes; specifically, a reluctance by pathologists to ascribe numeric probabilities but a willingness to reduce and standardize terminology. However, few schemes have been adopted, as evidenced by the continued investigation of these issues over the last 40 years.
The problem is not limited to human medicine. Several studies6–8,11 have been conducted to examine these issues with veterinary clinical pathologists. These studies observed a similar disconnect between clinical pathologists’ intended degree of certainty conveyed by various modifying terms and the interpretation of those modifying terms by clinicians. As with previous studies, participants in these studies were asked to choose a single numeric probability that they felt each term represented. The investigators then compared these single numeric values between clinical pathologists and clinicians. However, most people likely consider specific modifying terms to represent a range of probabilities rather than a single discrete numeric value. Furthermore, most studies have not asked what clinicians might prefer pathologists do when providing estimates of certainty.
Our specific objectives were to (1) examine the probability estimates for modifying terms used by clinical pathologists when interpreting cytologic samples and compare these to probability estimates assigned to these terms by clinicians and (2) provide restricted, standardizing terms used in cytology reports. Therefore, we conducted a series of surveys of clinical pathologists and veterinary clinicians to determine what ranges of probability various modifying terms cover, what terms appear largely synonymous (and therefore redundant), whether there is a disconnect in understanding the probability that each term conveys, and what degree of probability (based on the qualitative probability modifying term) would prompt a clinician to institute therapy, pursue additional confirmatory diagnostics, or discount the tentative diagnosis and pursue other differential diagnoses.
Materials and Methods
We created 2 surveys using an online survey platform14; one was provided to diplomates of the European College of Veterinary Clinical Pathology and the other to veterinarians who were members of the Veterinary Information Network (Supplementary Appendices S1 and S2). The surveys were voluntary and posed no known risks to respondents, so review board exemption was not sought. The surveys had some similar questions and some audience-specific questions. The survey scenario described a cytologic (fine-needle aspirate) sample obtained from a dog with suspected lymphoma. In both surveys, we asked participants to provide their own subjective estimates of numeric probabilities they associated with each of 17 qualitative probability modifying terms (and 1 situation in which no modifying term was used; ie, “diagnosis: lymphoma”). We used the same terms that had been used by Christopher et al,6,8 arranged in a haphazard order (ie, not in any decreasing or increasing likelihood or probability). These terms were as follows: “no evidence for,” “can’t rule out,” “presumptive of,” “possible,” “suggestive of,” “suspicious for,” “probable,” “supportive of,” “most likely,” “highly suspicious for,” “highly suggestive of,” “highly probable for,” “most consistent with,” “compatible with,” “indicative of,” “consistent with,” “diagnostic of,” and “definitive.”
We asked participants to select the low and high numeric values (grouped into 21 bins of 5% increments from 0% to 100%) that they felt represented the range of probabilities each term encompassed. If a participant believed that a single numeric value represented the term, they could choose a single option rather than a range. Additionally, if they did not use a term (as clinical pathologists) or did not interpret a term (as clinicians), participants had the option to choose “not applicable” for that term.
We then manually filled in the ranges between the limits provided by each participant; for example, if a participant selected 50% and 70% for a particular term, we added 55%, 60%, and 65%. We then summed each numeric probability bin across all participants for each term to create heat map–based histograms.
We also examined the ranges of probability that each term represented for clinical pathologists and clinicians by calculating the number of 5% probability bins chosen by each group. This would allow us to determine whether both groups had similar ranges of certainty regarding a term and whether the point estimates agreed. For example, if a clinician selected 40% to 80% for a particular term while a clinical pathologist selected 50% to 70% for the same term, both participants would have the same point estimate of certainty (60%) but the clinician would have a larger range of probabilities that they felt the term conveyed.
We then asked clinical pathologists to indicate their level of agreement (on a 5-point Likert scale) to the following series of questions:
Clinical pathologists should use a limited number of agreed-upon terms with predefined levels of certainty in their reports.
Clinical pathologists should provide a numeric range of certainty in their reports.
My clients (referring veterinarians) understand all the modifying terms that I and/or our team of consulting clinical pathologists use to denote a level of certainty in our reports.
I routinely recommend that the clinician call me to discuss the findings when modifiers suggest a high level of uncertainty (eg, “possible” or “suspicious for”).
I suspect that most clinicians routinely read only the interpretation summary or diagnosis line of the report.
We also asked clinicians to indicate their level of agreement (on a 5-point Likert scale) to the following series of questions:
Clinical pathologists should use a limited number of agreed-upon terms with predefined levels of certainty in their reports.
Clinical pathologists should provide a numeric range of certainty in their reports.
I understand all the modifying terms that my consulting clinical pathologists use to denote a level of certainty in their reports.
I routinely call the clinical pathologist to discuss the findings when modifiers suggest a high level of uncertainty (eg, “possible” or “suspicious for”).
I routinely read only the interpretation summary or diagnosis line of the report.
I would prefer a note from the clinical pathologist to contact them when modifiers suggest a high level of uncertainty.
We asked clinicians whether they requested specific pathologists. Finally, we asked clinicians whether they would likely institute therapy, pursue additional confirmatory diagnostics, or exclude lymphoma from further consideration for each probability modifying term.
The surveys were available from March 17, 2022, through April 5, 2022, for clinical pathologists and from April 19, 2022, through May 4, 2022, for clinicians. A follow-up email message was sent 2 weeks after the initial invitation to each group.
Statistical analysis
To determine whether the frequency distributions for each term differed between clinicians and clinical pathologists, we compared the distributions using 2-sample Kolmogorov-Smirnov tests. We adjusted the α values for these 18 comparisons to an α of .005 for each term.
To examine whether different terms covered essentially the same distributions, we visually examined the heat maps and grouped the terms on the basis of subjective estimates. We then compared the frequency distributions provided by clinical pathologists’ responses of select pairs of terms within each group to determine whether the frequency distributions of these terms differed using 2-sample Kolmogorov-Smirnov tests. We did not adjust the nominal α values but retained an α value of .05 for each comparison.
To examine whether clinicians assigned wider or narrower ranges of probability to each term than did clinical pathologists, we counted the number of bins assigned by each respondent for each term and created frequency distributions, ranging from 1 to 20, where 1 would represent that the respondent assigned only one 5% bin to the term, 5 would indicate that the respondent assigned five 5% bins to the term, etc. We then compared the resultant frequency distributions using 2-sample Kolmogorov-Smirnov tests with an α value adjusted to .005 for each comparison.
All statistical analyses were conducted with an add-on package15 to the spreadsheet software (Excel version 2211; Microsoft Corp).
For the remaining questions, we provided divergent bar charts to display the findings without any inferential statistical analyses.
Results
Four hundred sixty-six clinicians and 49 clinical pathologists completed the 2 surveys. The 466 clinicians consisted of 407 first-opinion clinicians, 31 referral clinicians, 14 emergency clinicians, 9 academic clinicians, and 5 shelter medicine clinicians. All clinical pathologists were diplomates of the European College of Veterinary Clinical Pathology.
When asked whether clinicians requested specific clinical pathologists to evaluate their cytology samples, 377 of 466 stated that they do not; only 28 stated that they usually or always do so.
We visually identified 7 broad categories of terms that were largely synonymous based on frequency heat map comparisons (Figure 1). Of the 18 terms, we found differences in the frequency distributions between clinicians and clinical pathologists for the numeric probabilities for 6 terms: “presumptive of,” “possible,” “probable,” “highly probable,” “consistent with,” and “diagnostic of” (P < .005 for all comparisons; Figure 2). We could not identify a consistent direction of disagreement for these terms; clinicians rated the probability lower than clinical pathologists for 3 and higher for 3 of these terms.
When comparing the frequency distributions generated from the responses of just the clinical pathologists, we found few differences in these distributions within our visually assigned groups. Specifically, “suggestive of” and “suspicious for” differed within their assigned group of 4 terms, as did “consistent with” and “indicative of” within their assigned group of 7 terms (P < .001 for all comparisons; Figure 1).
The range of numeric probabilities assigned to each term by clinicians mostly did not differ from the range provided by clinical pathologists. The 4 terms that appeared indicative of the highest degree of probability (“no modifier,” “diagnostic of,” “definitive for,” and “consistent with”) had wider frequency distributions for responses by clinicians than by clinical pathologists (P < .001 for each comparison; Figure 3).
Clinicians theoretically applied these 18 terms differently (Figure 4). When asked for each term whether they would feel comfortable instituting treatment for a putative canine lymphoma, most clinicians (between 393 and 408/466) responded that they would for “no modifier,” “diagnostic of,” and “definitive for.” However, only 60% (280/466) stated that they would do so for “consistent with,” a term that clinical pathologists ranked as having similar probability estimates to the other 3 terms. Similarly, 58% (270/466) of clinicians stated that they would institute therapy for the term “indicative of,” a term to which clinical pathologists applied a lower degree of probability than “consistent with.” More than 90% (411/442) of clinicians interpreted the term “no evidence for” as effectively ruling out the diagnosis of lymphoma. For all of the other terms, most clinicians would pursue additional diagnostics.
When asked whether clinical pathologists should use a limited range of probability modifying terms for cytology reports, approximately 80% (372/466) of clinicians and 85% (41/48) of clinical pathologists agreed or strongly agreed with this proposition (Figure 5).
However, when asked whether clinical pathologists should apply numeric ranges of probability along with or instead of probability modifying terms, more than half of clinicians (243/461) but only slightly more than a quarter of clinical pathologists (14/49) agreed or strongly agreed with this proposition (Figure 5).
When asked whether clinicians understand all the probability modifying terms that the clinical pathologists use, approximately one-third of clinical pathologists agreed (18/49) but a similar percentage (21/49) disagreed with this proposition. Similarly, approximately one-third of clinicians agreed (171/465) but one-third (164/465) disagreed with this proposition (Figure 5).
When asked whether they routinely call clinical pathologists to discuss terms describing high degrees of uncertainty (eg, “possible” or “suspicious for”), approximately one-third (179/465) stated that they do but one-third (181/465) stated they do not. However, approximately two-thirds of clinical pathologists (30/48) stated that they routinely recommend that clinicians call when the report contains such terms (Figure 5). Approximately three-quarters of clinicians (366/465) stated that they would prefer a note from the clinical pathologist to call when the clinical pathologist uses such terms in the report.
When asked whether they mostly read the interpretative summary and diagnosis line of cytology reports, 90% of clinical pathologists (44/49) suspected that clinicians routinely limit their reading to these sections. Conversely, only 20% of clinicians (90/465) agreed with this statement (Figure 5).
Discussion
Our study reinforced and expanded on the observations made recently by other investigators, supporting the need for standardization of terminology used by clinical pathologists to denote subjective probabilities when commenting on cytologic samples. Our study demonstrated that approximately 6 to 8 terms likely cover the ranges of probabilities that clinical pathologists perceive. Somewhat surprisingly, clinicians interpreted several of these terms similarly to clinical pathologists but, in some cases, perceived that the terms cover a larger range than that intended by clinical pathologists.
Our study suggested a redundancy with several probability terms, which appear to denote an essentially identical degree of uncertainty (or certainty). Both clinical pathologists and clinicians mostly agreed that reducing the number of terms would benefit the clarity of communication. Our observation that only one-third (171/466) of clinicians felt that they understood all the modifying terms used by clinical pathologists underscores the value of reducing the number of terms and better defining them. The decision to eliminate certain terms should initially examine agreement between the clinical pathologists and clinicians; those terms that denote similar degrees of uncertainty to clinical pathologists but are interpreted differently by clinicians are obvious candidates for elimination. For example, when examining the comparative heat maps, which show the modal probability estimates and ranges of probabilities of both groups, clinical pathologists assigned similar ranges to the terms “presumptive of” and “possible” (approx 50% to 60%). However, although clinicians assigned different degrees of probability to these terms, their estimates more closely matched those of the clinical pathologists for the term “possible” (40% to 60%) than for the term “presumptive of” (80% to 90%). Therefore, when choosing a term that represents a probability of 50%, “possible” appears to confer closer agreement than “presumptive of” to both groups.
Similarly, clinicians assigned higher probability estimates to the term “probable” (70% to 90%) than did clinical pathologists (60% to 80%). They also assigned higher probability estimates to the term “highly probable for” but lower probability estimates to the term “consistent with” than did clinical pathologists (Figure 1). Finally, for the 3 terms that denoted almost 100% probability for clinical pathologists (“diagnostic of,” “definitive,” and no modifier), clinicians ascribed wider ranges of probability to the terms “diagnostic of” and “definitive” than for the lack of any modifying term. Such differences in interpretation warrant eliminating these terms in favor of terms that denote similar degrees of probability or certainty but with closer agreement between clinicians and clinical pathologists.
Using this approach, we selected the 7 terms (1 from each category) that would cover the range of probability estimates with the least amount of overlap. Using the probability terms we included in our study, we would suggest the following terms be used to cover the ranges of probability estimates (in increasing order of probability): “no evidence for” (range, 0% to 20%), “can’t rule out” (range, 20% to 50%), “possible” (range, 50% to 65%), “suspicious for” (range, 65% to 75%), “most likely” (range, 75% to 85%), “indicative of” (range, 85% to 95%), and no modifier (range, 95% to 100%). These terms correspond to the range of probabilities covered by a nonoverlapping plurality of responses, providing clinical pathologists with complete, nonoverlapping coverage. Our terms should be viewed as a starting point, and clinical pathologists could be asked to agree upon other terms to represent these (or similar) ranges by means of an anonymous survey to obtain consensus on accepted, limited terminology.
A further issue is the use of different modifying terms within the interpretive section of a report and the commentary provided by the clinical pathologist. In most cases, the term used in the interpretive section could also be used in the commentary. The exception is the almost certain probability of a diagnosis, when the clinical pathologist uses no modifier term. In this case, something like “definitely has (disease or condition)” might be appropriate.
Our study found that clinicians would appreciate a numeric probability estimate to accompany the modifier term. The terms above provide a template for both clinicians and clinical pathologists to use to increase agreement or understanding in interpretation. Clinical pathologists, however, appeared mostly reluctant to do this, often citing lack of objective data to support their estimates. However, our study clearly demonstrated that they have subjective estimates of probability, based on their experience and limited scientific evidence, because they all provided numeric ranges for each term. Two clinical pathologists might differ in the probability term they would apply to the same cytologic sample, but that doesn’t detract from the use of both the term and a probability key that would apply to a particular clinical pathologist examining a particular sample and enhance communication between the reporting pathologist and the clinician.
We could not explain the disconnect in communication between clinical pathologists and clinicians about suggesting direct (phone) discussions for diagnoses carrying a high level of uncertainty. Only one-third (179/466) of clinicians call pathologists to discuss such results. Conversely, two-thirds (30/48) of clinical pathologists claimed that they advise clinicians to call about such results. However, three-quarters (366/466) of clinicians stated that they would prefer a comment to call if the clinical pathologist expressed a high level of uncertainty. We cannot reconcile these disparities. Nevertheless, clinical pathologists could improve interpretation of highly uncertain findings by routinely requesting that clinicians call about such results to discuss the specific findings.
Most clinical pathologists suspected that clinicians only read the interpretation or diagnosis portions of reports (where the modifying terms mostly appear). However, only 20% (90/466) of clinicians stated that they do this; most argued that they read the entire report, including the cytologic description.
Our findings were consistent with several studies by Christopher et al, who found substantial overlap (redundancy) in terms denoting probabilities of certainty and that the lack of a modifier term had a high accuracy for the diagnosis of lymphoma on cytologic samples.8 Furthermore, we found that at least 60% (≥ 280/466) of clinicians would be willing to institute antineoplastic therapy for diagnoses carrying the modifiers “consistent with,” “diagnostic of,” “indicative of,” and “definitive for” or no modifying term. These results partly agreed with those of Christopher et al,8 who found that 50% of clinicians would institute treatment for the probability term “consistent with.” Similarly, we found that most clinicians would pursue other diagnoses if the clinical pathologist used the term “no evidence for.” However, between 5% (38/462) and 50% (207/463) of clinicians might institute antineoplastic treatment for cases described with less-certain modifier terms. Such claims support our suggestions for a reduced and better-defined list of probability modifiers.
Our study differed slightly from those of Christopher et al6,8 in that we allowed participants to identify the range of probabilities that any term might encompass. However, our findings largely mirror those of the previous studies. We also used almost identical surveys for both groups of participants and conducted them contemporaneously.
Our study had a few limitations. We only invited clinical pathologists who were diplomates of the European College of Veterinary Clinical Pathology; therefore, most clinical pathologists resided in Europe and might use languages other than English to denote probability estimates.
However, all participants appeared to understand and use most of the terms (presumably in English or a translated version). Nevertheless, the similarity of findings to previous studies suggests that our results are representative of clinical pathologists at large.
The American Society of Veterinary Clinical Pathology Quality Assurance and Laboratory Standards Committee provides guidelines in a variety of quality topics in laboratory testing.16 We suggest that a working group of this committee might provide guidelines for standardization of terminology used in cytology reporting and the probability ranges associated with these. Adoption of such a guideline by clinical pathologists currently reporting cytology and training residents in cytologic interpretation would help establish more consistent use of a limited number of modifying terms and associated probabilities. Our study provides the impetus and a framework around which such a committee could build their guidelines.
In conclusion, we confirmed and expanded on previous studies of probability modifying terminology for cytologic samples. Our findings suggest considerable redundancy in terminology, with certain terms conveying greater disagreement than other terms between clinical pathologists providing the term and clinicians interpreting the term. Where such disagreement and redundancy exist, terms should be eliminated to avoid misinterpretation. We have provided a suggested list of terms that appear to span the ranges of probabilities and suggest that clinical pathologists might both restrict their probability modifiers to these terms and provide a numeric range of probabilities the term encompasses. This should improve communication between clinical pathologists and clinicians and, hopefully, improve patient care.
Supplementary Materials
Supplementary materials are posted online at the journal website: avmajournals.avma.org
Acknowledgments
No external funding was used in this study. The authors declare that there were no conflicts of interest.
References
- 1.↑
Nakao MA, Axelrod S. Numbers are better than words. Verbal specifications of frequency have no place in medicine. Am J Med. 1983;74(6):1061-1065. doi:10.1016/0002-9343(83)90819-7
- 2.
Kong A, Barnett GO, Mosteller F, Youtz C. How medical professionals evaluate expressions of probability. N Engl J Med. 1986;315(12):740-744. doi:10.1056/NEJM198609183151206
- 3.↑
Timmermans D. The roles of experience and domain of expertise in using numerical and verbal probability terms in medical decisions. Med Decis Making. 1994;14(2):146-156. doi:10.1177/0272989X9401400207
- 4.
Powsner SM, Costa J, Homer RJ. Clinicians are from Mars and pathologists are from Venus. Arch Pathol Lab Med. 2000;124(7):1040-1046. doi:10.5858/2000-124-1040-CAFMAP
- 5.
Gordon GH, Joos SK, Byrne J. Physician expressions of uncertainty during patient encounters. Patient Educ Couns. 2000;40(1):59-65. doi:10.1016/s0738-3991(99)00069-5
- 6.↑
Christopher MM, Hotz CS. Cytologic diagnosis: expression of probability by clinical pathologists. Vet Clin Pathol. 2004;33(2):84-95. doi:10.1111/j.1939-165x.2004.tb00354.x
- 7.
Christopher MM, Hotz CS, Shelly SM, Pion PD. Use of cytology as a diagnostic method in veterinary practice and assessment of communication between veterinary practitioners and veterinary clinical pathologists. J Am Vet Med Assoc. 2008;232(5):747-754. doi:10.2460/javma.232.5.747
- 8.↑
Christopher MM, Hotz CS, Shelly SM, Pion PD. Interpretation by clinicians of probability expressions in cytology reports and effect on clinical decision-making. J Vet Intern Med. 2010;24(3):496-503. doi:10.1111/j.1939-1676.2010.0492.x
- 9.
Lindley SW, Gillies EM, Hassell LA. Communicating diagnostic uncertainty in surgical pathology reports: disparities between sender and receiver. Pathol Res Pract. 2014;210(10):628-633. doi:10.1016/j.prp.2014.04.006
- 10.
Newman AW, Rishniw M, Behling-Kelly E. Reporting and interpreting red blood cell morphology: is there discordance between clinical pathologists and clinicians? Vet Clin Pathol. 2014;43(4):487-495. doi:10.1111/vcp.12202
- 11.↑
Christopher MM, Ku CK. Likelihood of neoplasia for diagnoses modified by probability terms in canine and feline lymph node cytology: how probable is probable? Front Vet Sci. 2018;5:246. doi:10.3389/fvets.2018.00246
- 12.↑
Beyth-Marom R. How probable is probable? A numerical translation of verbal probability expressions. J Forecast. 1982;1(3):257-269. doi:10.1002/for.3980010305
- 13.↑
Renooij S, Witteman C. Talking probabilities: communicating probabilistic information with words and numbers. Int J Approx Reason. 1999;22(3):169-194. doi:10.1016/S0888-613X(99)00027-4
- 15.↑
Zaiontz C. Real Statistics using Excel. Real Statistics. Accessed January 9, 2023. https://www.real-statistics.com/
- 16.↑
ASVCP Quality Assurance and Laboratory Standards Guidelines. American Society for Veterinary Clinical Pathology. Accessed October 27, 2022. https://www.asvcp.org/page/QALS_Guidelines