Evaluation of interobserver agreement for postoperative pain and sedation assessment in cats

Javier Benito Department of Clinical Sciences, Faculty of Veterinary Medicine, University of Montreal, Saint-Hyacinthe, QC J2S 2M2, Canada.

Search for other papers by Javier Benito in
Current site
Google Scholar
PubMed
Close
 DVM, PhD
,
Beatriz P. Monteiro Department of Veterinary Biomedicine, Faculty of Veterinary Medicine, University of Montreal, Saint-Hyacinthe, QC J2S 2M2, Canada.

Search for other papers by Beatriz P. Monteiro in
Current site
Google Scholar
PubMed
Close
 DVM
,
Guy Beauchamp Department of Office of the Associate Dean for Research and Graduate Studies, Faculty of Veterinary Medicine, University of Montreal, Saint-Hyacinthe, QC J2S 2M2, Canada.

Search for other papers by Guy Beauchamp in
Current site
Google Scholar
PubMed
Close
 PhD
,
B. Duncan X. Lascelles Comparative Pain Research Laboratory & Center for Comparative Medicine and Translational Research, College of Veterinary Medicine, University of North Carolina, Raleigh, NC 27606.
Center for Pain Research and Innovation, University of North Carolina, Chapel Hill, NC 27599.

Search for other papers by B. Duncan X. Lascelles in
Current site
Google Scholar
PubMed
Close
 BVSc, PhD
, and
Paulo V. Steagall Department of Clinical Sciences, Faculty of Veterinary Medicine, University of Montreal, Saint-Hyacinthe, QC J2S 2M2, Canada.

Search for other papers by Paulo V. Steagall in
Current site
Google Scholar
PubMed
Close
 DVM, PhD

Abstract

OBJECTIVE To evaluate agreement between observers with different training and experience for assessment of postoperative pain and sedation in cats by use of a dynamic and interactive visual analog scale (DIVAS) and for assessment of postoperative pain in the same cats with a multidimensional composite pain scale (MCPS).

DESIGN Randomized, controlled, blinded study.

ANIMALS 45 adult cats undergoing ovariohysterectomy.

PROCEDURES Cats received 1 of 3 preoperative treatments: bupivacaine, IP; meloxicam, SC with saline (0.9% NaCl) solution, IP, (positive control); or saline solution only, IP (negative control). All cats received premedication with buprenorphine prior to general anesthesia. An experienced observer (observer 1; male; native language, Spanish) used scales in English, and an inexperienced observer (observer 2; female; native language, French) used scales in French to assess signs of sedation and pain. Rescue analgesia was administered according to MCPS scoring by observer 1. Mean pain and sedation scores per treatment and time point, proportions of cats in each group with MCPS scores necessitating rescue analgesia, and mean MCPS scores assigned at the time of rescue analgesia were compared between observers. Agreement was assessed by intraclass correlation coefficient determination. Percentage disagreement between observers on the need for rescue analgesia was calculated.

RESULTS Interobserver agreements for pain scores were good, and that for sedation scores was fair. On the basis of observer 1's MCPS scores, a greater proportion of cats in the negative control group received rescue analgesia than in the bupivacaine or positive control groups. Scores from observer 2 indicated a greater proportion of cats in the negative control group than in the positive control group required rescue analgesia but identified no significant difference between the negative control and bupivacaine groups for this variable. Overall, disagreement regarding need for rescue analgesia was identified for 22 of 360 (6.1%) paired observations.

CONCLUSIONS AND CLINICAL RELEVANCE Interobserver differences in assessing pain can lead to different conclusions regarding treatment effectiveness.

Abstract

OBJECTIVE To evaluate agreement between observers with different training and experience for assessment of postoperative pain and sedation in cats by use of a dynamic and interactive visual analog scale (DIVAS) and for assessment of postoperative pain in the same cats with a multidimensional composite pain scale (MCPS).

DESIGN Randomized, controlled, blinded study.

ANIMALS 45 adult cats undergoing ovariohysterectomy.

PROCEDURES Cats received 1 of 3 preoperative treatments: bupivacaine, IP; meloxicam, SC with saline (0.9% NaCl) solution, IP, (positive control); or saline solution only, IP (negative control). All cats received premedication with buprenorphine prior to general anesthesia. An experienced observer (observer 1; male; native language, Spanish) used scales in English, and an inexperienced observer (observer 2; female; native language, French) used scales in French to assess signs of sedation and pain. Rescue analgesia was administered according to MCPS scoring by observer 1. Mean pain and sedation scores per treatment and time point, proportions of cats in each group with MCPS scores necessitating rescue analgesia, and mean MCPS scores assigned at the time of rescue analgesia were compared between observers. Agreement was assessed by intraclass correlation coefficient determination. Percentage disagreement between observers on the need for rescue analgesia was calculated.

RESULTS Interobserver agreements for pain scores were good, and that for sedation scores was fair. On the basis of observer 1's MCPS scores, a greater proportion of cats in the negative control group received rescue analgesia than in the bupivacaine or positive control groups. Scores from observer 2 indicated a greater proportion of cats in the negative control group than in the positive control group required rescue analgesia but identified no significant difference between the negative control and bupivacaine groups for this variable. Overall, disagreement regarding need for rescue analgesia was identified for 22 of 360 (6.1%) paired observations.

CONCLUSIONS AND CLINICAL RELEVANCE Interobserver differences in assessing pain can lead to different conclusions regarding treatment effectiveness.

The assessment of pain in animals is paramount to ensure its adequate treatment, and currently this involves the use of composite or multicomposite scoring systems to assess signs of pain.1 Ideally, pain scales would have content, construct, and criterion validity; interrater and intrarater reliability; responsiveness; and a defined, validated cutoff point for intervention by analgesic administration.2 The potential for cross-cultural differences makes it important to validate a given written pain scale instrument for use in different languages, considering that translation, back-translation, and determination of semantic equivalence must be performed. The UNESP-Botucatu MCPS is an instrument used for assessment of acute pain after ovariohysterectomy in cats. This pain scale was originally developed in Portuguese and has been recently translated into English, Spanish, and French and validated in each of these languages.2–6

Several studies7–9 have investigated interobserver differences in postoperative pain assessment for human patients to evaluate the ability of healthcare professionals to reliably perceive and treat pain. In 1 study,9 health professionals consistently scored pain in children lower than did patients and their guardians. Furthermore, prior knowledge of the analgesic regimen or a preconceived idea of postprocedural pain could influence an observer's perception of signs of pain in a patient.9 To the authors' knowledge, no studies that have examined agreement among different observers using the UNESP-Botucatu MCPS for assessment of postoperative pain in cats have been published.

The objective of the study reported here was to evaluate interobserver agreement for postoperative pain and sedation scoring of cats following ovariohysterectomy as determined by use of a DIVAS and for pain scoring of the same cats by use of an MCPS written and validated in English or French. We hypothesized that pain scores determined with each scoring instrument would not differ significantly between 2 observers of different genders and experience levels using a DIVAS and an MCPS written in English or French. Data on the analgesic effects of IP administration of bupivacaine in the cats of this study were gathered concurrently and have been published elsewhere.10 The study reported here provides additional data on postoperative pain scores in the same cats as assessed by multiple investigators.

Materials and Methods

Animals

Forty-five client-owned, healthy, mixed-breed cats were scheduled for elective ovariohysterectomy in a prospective, randomized, controlled, blinded clinical trial.10 Cats were housed individually in the cat ward of a veterinary teaching hospital. All study cats were considered healthy on the basis of medical history, results of a complete physical examination, and assessment of Hct and serum total protein concentration. Cats were excluded if they were obese (body condition score > 7 [scale, 1 to 9]) or had cardiac arrhythmias, anemia, or other clinical signs of disease.10 Aggressive behaviors, pregnancy, and lactation were also causes for exclusion.10 Informed consent was obtained from cat owners prior to their enrollment in the study. The study protocol was approved by the animal care committee of the Faculty of Veterinary Medicine of the University of Montreal (14-Rech-1744).

Anesthesia and surgery

Details of the surgery have been described elsewhere.10 Briefly, cats were premedicated with acepromazinea (0.05 mg/kg [0.023 mg/lb], IM) and buprenorphineb (0.01 mg/kg [0.005 mg/lb], IM). Anesthesia was induced with propofolc (IV, to effect) and maintained with isofluraned in oxygen delivered through a nonrebreathing circuit. Cats were positioned in dorsal recumbency on a warming blanket. Monitoringe (ECG, capnography, measurement of inspired and expired concentrations of isoflurane, pulse oximetry, and esophageal temperature measurement) was performed throughout the procedure. Blood pressure was measured with a Doppler ultrasonic flow detector. Lactated Ringer solution (10 mL/kg/h, IV) was administered throughout surgery.

Following aseptic preparation of the surgical site, assigned study treatments were administered, and after a 2-minute waiting period, ovariohysterectomy was performed by one of the authors (BPM). A 3-cm ventral midline incision was made, and a modified 3-clamp ovariohysterectomy technique was used, followed by routine closure of the abdominal wall and skin. After surgery, cats were monitored routinely during recovery. Extubation was performed when the cats' palpebral reflexes were evident.

Treatment groups

Cats were randomly assignedf to 3 treatment groups as part of the previously published investigation (n = 15/group).10 One group received bupivacaine hydrochlorideg (2 mg/kg [0.9 mg/lb], IP). Another group received meloxicamh (0.2 mg/kg [0.09 mg/lb], SC) and saline (0.9% NaCl) solutioni (volume equivalent to that of bupivacaine, IP) as a positive control treatment, and the remaining group received saline solutioni only (volume equivalent to that of bupivacaine, IP) as a negative control treatment. For IP injections, a 3-mL syringe was attached to a 22-gauge IV catheter, and the assigned solution was instilled into the peritoneal space over the right and left ovarian pedicles and over the caudal aspect of the uterine body.

Pain and sedation scores

Pain and sedation assessments were completed by 2 observers who were blinded to the analgesic treatment groups of the patients. Observer 1 was a male second-year resident in veterinary anesthesia and analgesia who had previous experience in feline pain assessment (JB).10 This individual (native language, Spanish) used pain scales written in English (the DIVAS did not include text, but forms for recording the data were in the same language as the MCPS that was used). Rescue analgesia, when needed, was administered on the basis of assessments by observer 1. Observer 2 was a female veterinary student (native language, French) participating in the final year of a 5-year DVM program; this observer used the same pain scales as observer 1, except that the documents were written in French. Observer 2 did not have previous experience in pain and sedation evaluation of veterinary patients; assessments made by this individual were part of a summer student training program in feline pain management and were not used in the concurrently performed study10 involving the same cats.

In brief, both observers received 2 hours of instruction on the theory of feline pain assessment from a board-certified anesthesiologist (PVS). Both observers then completed approximately 6 hours of video training in feline pain assessment with the UNESP-Botucatu MCPS11 prior to assessment of study cats.

Pain and sedation were evaluated with a 100-mm (10-cm) DIVAS,12–15 on which 0 cm represented no signs of pain or no sedation, and 10 cm represented signs of the worst pain or deepest sedation that the observer imagined could ever arise from the ovariohysterectomy procedure. Pain was also evaluated with the UNESP-Botucatu MCPS.2,6 The DIVAS evaluation was always recorded before MCPS assessment. The MCPS included 3 subscales (pain expression, psychomotor change, and physiologic variables), with a total score from 0 (no signs of pain) to 30 (greatest signs of pain); however, 1 physiologic variable (blood pressure monitoring) was omitted, and the range of potential scores according for the scale as used in the study was 0 to 27. Cats were initially evaluated inside their cages without being disturbed, and then they were gently handled and encouraged to move. The abdominal area and surgical wound were palpated and the scores for each domain were assigned according to the instructions.2,6,10 Assessments were performed concurrently but independently. Palpation was performed by observer 1 and then by observer 2, with an interval of 2 minutes between these assessments.

Data collection and rescue analgesia

Evaluations were performed 60 minutes prior to the induction of anesthesia (time 0) and at 0.5, 1, 2, 3, 4, 6, and 8 hours after surgery (with the time of extubation considered the end of surgery). Rescue analgesia was provided for any cat with an MCPS score ≥ 6 as determined by observer 1.10 The rescue treatment for all cats included buprenorphine (0.02 mg/kg [0.009 mg/lb], IV); cats that had received bupivacaine or saline solution only as study treatments immediately prior to surgery were also given meloxicam (0.2 mg/kg, SC).10 Data collected after the administration of rescue analgesia were not included in the statistical analysis,10 but cats continued to be assessed up to 8 hours after surgery. The score used as a cutoff for rescue analgesia (> 5) was lower than the cutoff score of > 7 used with the original scale in English2 because blood pressure measurements were not performed.

Postoperative treatments were administered by 2 investigators who were not involved with pain assessment. The scores assigned to each cat by each observer per time point were recorded for all scales used. The number and group assignments of cats with MCPS scores at or above the cutoff for rescue analgesia and the postoperative time points at which these scores were assigned were recorded for each observer. Determination of the need for rescue analgesia was considered to be the most important outcome for comparison because a main finding in the previously published study10 of these same cats, that bupivacaine provided acceptable postoperative analgesia, was based on the results of statistical comparisons for this factor among groups. We wished to investigate how assessments by another observer with different background and experience, using the same scales in a different language, could influence such results.

Statistical analysis

Results of a Shapiro-Wilk test indicated that the data were normally distributed. Interobserver agreement of DIVAS scores for pain, DIVAS scores for sedation, and MCPS scores for each cat at all time points (excluding data after rescue analgesia) was assessed by the ICC obtained from a mixed linear model. The results were interpreted by the Altman classification scheme as follows: 0.81 to 1.00 = very good, 0.61 to 0.80 = good, 0.41 to 0.60 = moderate, 0.21 to 0.40 = fair, and < 0.2 = poor,16 with values converted to percentages for reporting purposes. Means of DIVAS pain scores, DIVAS sedation scores, and MCPS scores from each observer were compared for each treatment and for each time period. Means were compared and analyzed at each time point separately by use of a repeated-measures linear model with observer as a within-subject factor and treatment group of the cat as a between-subject factor. The subject was the individual cat receiving treatment. A simple Bonferroni adjustment for multiple comparisons was performed for these analyses, and values of P ≤ 0.0167 (0.05/3) were considered significant. Sample size was determined by calculations performed for the ability to detect a difference in treatment failure rates in the concurrent study.10

The percentage of disagreement between observers regarding rescue analgesia (ie, the result of pairwise comparisons in which one observer, but not the other, recorded a score ≥ 6 on the MCPS [indicating the need for rescue analgesia]) was calculated. Furthermore, the number of cats with scores ≥ 6 (per each observer) was analyzed with the Cochran-Mantel-Haenszel test for ordinal variables followed by pairwise comparisons between analgesic treatment groups, and the MCPS rating scores from each observer at the time of the rescue analgesia (for cats that received the treatment on the basis of observer 1 evaluations) were compared and analyzed for agreement. First, a concordance correlation coefficient was calculated by use of a scatterplot. Then, the difference between scores from the 2 observers was plotted against the mean of those scores by the Bland-Altman method. Statistical analyses were performed using standard software.j Concordance and agreement plots were carried out with another standard software package.k

Results

Pain scores

All cats were examined at each time point by both observers; however, the number of cats in each group that were included in the analyses varied owing to exclusion following rescue analgesia. At time 0 and at 0.5 and 1 hour after surgery, all cats (15/group) were included in the analysis. For the positive control group, 13 cats were included at all subsequent time points (2, 3, 4, 6, and 8 hours). For the negative control group, there were 11, 8, 5, 4, and 3 cats included at 2, 3, 4, 6, and 8 hours after surgery, respectively. For the bupivacaine treatment group, there were 13 cats included at 2 hours, 12 at 3 hours, and 11 at 4, 6, and 8 hours after surgery.

The ICCs were 65.4% (95% CI, 19.20% to 80.22%) and 70.7% (95% CI, 21.75% to 89.02%) for DIVAS pain scores and MCPS scores, respectively, indicating good agreement between observers. Mean MCPS scores assessed by observer 2 were significantly lower than those determined by observer 1 for cats in the positive control group 0.5 (P < 0.0001), 1 (P = 0.0003), 2 (P = 0.0020), 3 (P = 0.0135), and 4 (P = 0.0032) hours after surgery and for cats in the negative control group at 0.5 (P = 0.0089) and 1 (P = 0.0027) hour after surgery (Figure 1). For the bupivacaine treatment group, mean baseline MCPS scores recorded by observer 2 were significantly (P = 0.0055) higher than those recorded by observer 1, but there were no significant differences between mean observer scores for these cats at other time points. Mean DIVAS pain scores were not significantly different between observers at any time point for cats in the positive and negative control groups; however, for cats in the bupivacaine treatment group, the mean scores determined by observer 2 were significantly higher than those determined by observer 1 at 4 (P = 0.0087) and 8 (P = 0.0125) hours after surgery (Figure 2).

Figure 1—
Figure 1—

Mean ± SEM MCPS scores assigned by 2 observers to 45 healthy cats that underwent ovariohysterectomy under general anesthesia. Cats were randomly assigned to receive 1 of 3 preoperative treatments: bupivacaine (2 mg/kg [0.9 mg/lb], IP; white bars); meloxicam (0.2 mg/kg [0.09 mg/lb], SC) with saline (0.9% NaCl) solution, IP (positive control; black bars); or saline solution only, IP (negative control; striped bars). All cats had received premedication that included buprenorphine (0.01 mg/kg [0.005 mg/lb], IM). An experienced observer (observer 1; male; native language, Spanish) used scales written in English and an inexperienced observer (observer 2; female; native language, French) used scales written in French to assess signs of sedation (with a DIVAS) and pain (with a DIVAS and the UNESP-Botucatu MCPS) at indicated time points. Baseline values were obtained before any drug administration, and remaining time points were measured from the end of surgery (the time of extubation). Possible MCPS scores ranged from 0 (no signs of pain) to 27 (greatest signs of pain); blood pressure evaluation, which is normally part of the MCPS, was not performed. Cats for which the experienced observer assigned MCPS scores ≥ 6 at any time point received rescue analgesia and were subsequently excluded from the analysis (total, 18 cats; 4, 2, and 12 in the bupivacaine, positive control, and negative control groups, respectively). *Within a time point, mean scores are significantly (P ≤ 0.0167 after Bonferroni correction) different between observers for the same treatment group. OBS = Observer.

Citation: Journal of the American Veterinary Medical Association 251, 5; 10.2460/javma.251.5.544

Figure 2—
Figure 2—

Mean ± SEM DIVAS pain scores assessed by observers 1 and 2 for the same cats as in Figure 1. Possible DIVAS scores ranged from 0 cm (no signs of pain) to 10 cm (signs of the worst pain that the observer imagined could arise from the ovariohysterectomy procedure). See Figure 1 for key.

Citation: Journal of the American Veterinary Medical Association 251, 5; 10.2460/javma.251.5.544

Sedation scores

The ICC for DIVAS sedation scores was 36.8% (95% CI, 11.07% to 69.15%), indicating fair interobserver agreement. Mean sedation scores assessed by observer 2 were significantly (P < 0.0001 for all comparisons) higher, compared with those assessed by observer 1, for all groups at 0.5 and 1 hour after surgery and for the bupivacaine treatment group 2 hours after surgery (Figure 3). Four hours after surgery, the mean sedation score for the negative control group assessed by observer 2 was significantly (P = 0.0145) lower than that assessed by observer 1.

Figure 3—
Figure 3—

Mean ± SEM DIVAS sedation scores assessed by observers 1 and 2 for the same cats as in Figure 1. Possible DIVAS scores ranged from 0 cm (no signs of sedation) to 10 cm (signs of the deepest sedation that the observer imagined could arise from the ovariohysterectomy procedure). See Figure 1 for key.

Citation: Journal of the American Veterinary Medical Association 251, 5; 10.2460/javma.251.5.544

Rescue analgesia

On the basis of MCPS scores assigned by observer 1, 18 cats received rescue analgesia.10 The percentage of cats with scores that warranted rescue analgesia did not differ significantly (P = 0.1553) between observers 1 (18/45 [40%]) and 2 (20/45 [44%]). When scores from both observers at all time points were considered, the total percentage disagreement regarding the need for rescue analgesia was 6.1% (ie, the decision of whether to administer rescue analgesia differed in 22/360 pairs of observations). When data for the 18 cats that had been given rescue analgesia were compared between observers, the outcome for rescue analgesia would have been different in 2.8% of cases (ie, in 4/144 paired observations, rescue analgesia would not have been administered on the basis of observer 2 scores). For observer 1, the proportion of cats with scores requiring rescue analgesia was significantly (P = 0.0005) higher in the negative control group (12/15), compared with the positive control group (2/15) and bupivacaine treatment group (4/15), and no difference between the latter 2 groups was observed (P = 0.6513). For observer 2, the proportion of cats with scores requiring rescue analgesia was significantly (P = 0.0047) higher in the negative control group (11/15) than in the positive control group (2/15); however, there was no significant (P = 0.1098) difference in this variable between the bupivacaine treatment (7/15) and negative control groups.

The concordance correlation coefficient (r) between MCPS scores from the 2 observers at the time of rescue treatment administration was 0.5413 (95% CI, 0.0996 to 0.8048) with a precision (Pearson ρ) of 0.5413 and accuracy (bias correction factor [Cb]) of 0.9834. The correlation between scores assessed by the 2 observers was significant (P = 0.020). The mean difference of scores at the intervention point for all cats that received rescue analgesia was 0.3 (95% limits of agreement, −3.3489 to 4.0155; Figure 4).

Figure 4—
Figure 4—

Bland-Altman plot of agreement between MCPS scores assigned by observers 1 and 2 for 18 cats that received rescue analgesia (on the basis of scores ≥ 6 as assigned by observer 1) at the time point when the determination for rescue analgesia was made. Some data points overlapped. The x-axis represents the mean of scores from the 2 observers, and the y-axis represents the difference between them. The solid black horizontal line indicates the mean difference (bias) between observer scores (0.3), and thick dashed black lines represent the upper and lower 95% limits of agreement (± 1.96 SD; 4.0 and −3.3, respectively). The dashed and dotted line crossing the mean difference line represents the line of regression, and gray curved lines indicate the 95% CI of the regression line. The mean ± SEM intercept coefficient for the line of regression was 0.8519 ± 1.9905, and the mean ± SEM slope coefficient was −0.07293 ± 0.2725. See Figure 1 for remainder of key.

Citation: Journal of the American Veterinary Medical Association 251, 5; 10.2460/javma.251.5.544

Discussion

In the present study, postoperative pain (MCPS and DIVAS) and sedation (DIVAS) scores assigned by 2 observers of different genders, training, and experience to cats that underwent ovariohysterectomy were evaluated for agreement. One observer used pain scales written in English (observer 1), and the other used the same scales written in French (observer 2). Rescue analgesia was performed on the basis of pain scores determined by observer 1, and those data were used to compare the 3 study treatments (positive control, negative control, and bupivacaine) in a previously published report.10 Although interobserver agreement in the present study for DIVAS pain scores and MCPS scores was good (ICCs, 65.4% and 70.7%, respectively), that for sedation scores was fair (ICC, 36.8%). The discrepancy in sedation scores between observers, with higher sedation scores for all groups recorded by observer 2 than by observer 1 during the first hour after surgery, could be explained by the lack of a validated scoring system for sedation in cats. In addition, the observers did not receive any video training specifically for assessment of sedation in cats.

The MCPS is a validated pain scale for evaluation of cats after ovariohysterectomy.2 Although interobserver agreement of these scores on the basis of ICCs was good, there were significant differences between observers' pain scores, with lower scores recorded by observer 2 for the positive and negative control groups 0.5 and 1 hour after surgery and for the positive control group through 4 hours after surgery. These results could be explained in part by higher DIVAS sedation scores that were recorded by observer 2 at the first 2 postoperative time points; it is possible that pain-associated behaviors could have been interpreted as signs of sedation, as has been reported in studies17,18 of dogs. However, the frequencies of cats with scores at or above the cutoff for rescue analgesia in the positive and negative control groups were similar for the 2 observers, and the lower MCPS scores assigned by observer 2 would not have changed this outcome in the previously published study.10 The differences in this study might only have reflected the subjectivity of pain assessment, even when using validated pain scales. Furthermore, the mean MCPS score for cats in the bupivacaine treatment group at baseline as assessed by observer 2 was significantly higher than that determined by observer 1. We considered it possible that stress induced by hospitalization could have been a confounding factor in the assessment of pain by different observers at this preoperative time point because drugs had not yet been administered, and we believed that the difference was likely clinically irrelevant.

Although the ICC for DIVAS pain scores indicated good agreement between observers, and there were no significant interobserver differences in scores for the positive and negative control groups at any time point, there were significant interobserver differences for DIVAS pain scores for the bupivacaine treatment group 4 and 8 hours after surgery. It is interesting that evaluation with the DIVAS pain scale, which has not been validated for this use, resulted in an overall level of interobserver agreement similar to that found for MCPS, particularly when considering that video training for pain assessment was only provided for the latter scale. Nevertheless, there may have been some inherent bias in assessments performed with the DIVAS. The DIVAS is more sensitive and reliable, compared with simple descriptive scales, when used by experienced observers to evaluate postoperative pain in small animals.17,19,20 However, the subjectivity of the scoring method likely influenced the interobserver differences found for the bupivacaine treatment group at some time points. Our results for the use of the DIVAS were generally in agreement with those of previous studies21,22 where VAS pain scale assessments of postoperative pain in dogs were not significantly different among observers. However, they were in disagreement with the results of other studies,20,23,24 in which observer scores were consistent over the time, but findings suggested that DIVAS scores might be subject to large interobserver variations and a high amount of variability in pain assessment.

The frequencies of cats with MCPS scores necessitating rescue analgesia (ie, ≥ 6) in the present study were similar between observers, with disagreement that would have influenced the decision to administer the additional treatment in only 22 of 360 (6.1%) paired observations. For both observers, the proportion of cats with scores at or above this cutoff was high in the negative control group (12/15 and 11/15 for observers 1 and 2, respectively). However, when data for the 18 cats that received rescue analgesia in the previously published study10 were considered, this outcome would have been different in 4 of 144 (2.8%) paired observations.

In that study,10 the proportion of cats that received rescue analgesia on the basis of MCPS scores assessed by observer 1 was significantly greater in the negative control group than in the bupivacaine or positive control treatment groups, with no difference between the latter 2 groups, and this led to the conclusion that IP bupivacaine treatment in cats premedicated with buprenorphine provided postoperative analgesia superior to that afforded by the premedicant buprenorphine alone in the previously published study.10 In the present study, however, the proportion of cats that would have received rescue analgesia did not differ between the negative control and bupivacaine treatment groups as assessed by observer 2. Thus, the results and conclusions drawn from that study10 would have been different if data from observer 2 had been considered for rescue analgesic administration. This highlights the importance of interobserver variability when critically evaluating the literature. It should be considered that some observer-dependent findings could potentially result in misleading or erroneous conclusions related to a new analgesic treatment or technique. Similar findings were observed in another study25 in which second-year veterinary students and veterinary technicians reported different postoperative pain scores for dogs. There is an intrinsic subjectivity in pain scoring for veterinary patients, because the scoring systems rely on an observer's interpretation of behavioral changes. We identified 3 main potential reasons for the differences in results between observers in the present study: use of the MCPS in different languages, one observer having had previous experience in pain assessment whereas the other did not, and gender-related differences. These were also the main limitations of the study, and it was not possible to identify the influence of these individual variables on the results of this study.

Previously, agreement between evaluators as measured by ICC varied from good to very good when the MCPS used in the present study was validated in various languages.2–6 In the present study, in addition to ICCs indicating good interobserver agreement for MCPS scores, calculation of the concordance correlation coefficient between observers at the time rescue analgesia administration revealed a significant correlation and good concordance. Taken together, these findings suggest that use of the MCPS in different languages might have minimal influence in the evaluation of postoperative pain following ovariohysterectomy in cats.

Results of a study1 involving dogs suggested that the agreement between students in their final year of veterinary school and board-certified anesthesiologists was poor for evaluation of postoperative pain in dogs, with 95% limits of agreement between anesthesiologists and students of −7.0 to 4.6 cm for the DIVAS and −13.0 to 9.0 for the short form of the Glagow Composite-Measure Pain Scale.1 In that study,1 most students were female, first- and second-year veterinary students, and overall, students assigned higher pain scores than did anesthesiologists. However, dogs assigned higher pain scores by anesthesiologists received lower scores when evaluated by students, and those assigned lower pain scores by anesthesiologists received higher scores from students.1 When interobserver differences were identified in mean MCPS scores of cats after surgery in the present study, the individual with previous experience in feline pain assessment had assigned higher pain scores than did the veterinary student.

Gender has been reported to have an important influence on pain assessment and analgesic administration in human medicine, and several studies26–29 involving nurses and physicians have reported biases with regard to the gender of the health care provider that may lead to overtreatment or undertreatment of reported pain. In some recent reports in the veterinary literature, women tended to score pain higher than did men in assessment of patients with a variety of painful conditions.30,31 Therefore, gender as well as training and experience of the observer can impact pain assessment and might have influenced the results of the present study.

In the study reported here, pain scoring data were excluded after the administration of rescue analgesia, and this approach may have impacted our findings. The analgesic benefit of a treatment could have been overestimated because higher pain scores might have been omitted, and analysis bias could have resulted. The loss of data leading to uneven group sizes also limited the analysis. However, group comparisons would be inappropriate if cats that had received rescue analgesia were compared with those that did not.

The results of the present study highlighted the importance of interobserver variability even when a validated pain scoring system is used and the interobserver agreement is good. Studies on pain assessment should include an experienced observer. This issue should be considered when critically evaluating the literature on the subject.

Acknowledgments

Supported by the Morris Animal Foundation; Fonds en Santé des Animaux de Compagnie, Zoetis Canada; and the Comparative Pain Research Laboratory, College of Veterinary Medicine, North Carolina State University. Dr. Beatriz Monteiro received support from the Vanier scholarship.

The authors declare that there were no conflicts of interest.

Presented in part as a poster communication at the 12th World Congress of Veterinary Anaesthesiology, Kyoto, Japan, September 2015.

The authors thank Dr. Anne-Marie Lavoie for technical assistance.

ABBREVIATIONS

CI

Confidence interval

DIVAS

Dynamic and interactive visual analog scale

ICC

Intraclass correlation coefficient

MCPS

Multidimensional composite pain scale

UNESP

Universidade Estadual Paulista

Footnotes

a.

Atravet acepromazine maleate USP, Boehringer Ingelheim, Burlington, ON, Canada.

b.

Vetergesic, Champion Alstoe Animal Health, York, England.

c.

Diprivan 1%, AstraZeneca Canada Inc, Mississauga, ON, Canada.

d.

Isoflurane USP, Pharmaceuticals Partners of Canada Inc, Richmond Hill, ON, Canada.

e.

Lifewindow 6000V veterinary multiparameter monitor, Digicare Animal Health, Boynton Beach, Fla.

f.

Research randomizer. Available at: www.randomization.com. Accessed May 5, 2014.

g.

Sensorcaine, bupivacaine HCl 0.5% USP, AstraZeneca Canada Inc, Mississauga, ON, Canada.

h.

Metacam 0.5%, Boehringer Ingelheim, Burlington, ON, Canada.

i.

0.9% Sodium chloride USP, Hospira, Montreal, QC, Canada.

j.

SAS, version 9.3, SAS Institute Inc, Cary, NC.

k.

MedCalc, version 12, MedCalc Software, Ostend, Belgium.

References

  • 1. Barletta M, Young CN, Quandt JE, et al. Agreement between veterinary students and anesthesiologists regarding postoperative pain assessment in dogs. Vet Anaesth Analg 2016; 43: 9198.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2. Brondani JT, Mama KR, Luna SP, et al. Validation of the English version of the UNESP-Botucatu multidimensional composite pain scale for assessing postoperative pain in cats. BMC Vet Res 2013; 9: 143.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 3. Brondani JT, Mama KR, Luna SP, et al. Validation of the Spanish version of the UNESP-Botucatu multidimensional composite pain scale to assess postoperative pain in cats. Arch Med Vet 2014; 46: 477486.

    • Search Google Scholar
    • Export Citation
  • 4. Brondani JT, Luna SP, Minto BW, et al. Validity and responsiveness of a multidimensional composite scale to assess postoperative pain in cats. Arq Bras Med Vet Zootec 2012; 64: 15291538.

    • Search Google Scholar
    • Export Citation
  • 5. Brondani JT, Luna SP, Minto BW, et al. Reliability and cut-off point related to analgesic intervention of a multidimensional composite scale to assess postoperative pain in cats. Arq Bras Med Vet Zootec 2013; 65: 153162.

    • Search Google Scholar
    • Export Citation
  • 6. Steagall PV, Monteiro BP, Lavoie AM, et al. Validation de la version francophone d'une échelle composite multidimensionnelle pour l'évaluation de la douleur postopératoire chez les chats. Can Vet J 2017; 58: 5664.

    • Search Google Scholar
    • Export Citation
  • 7. Khin Hla T, Hegarty M, Russell P, et al. Perception of pediatric pain: a comparison of postoperative pain assessments between child, parent, nurse, and independent observer. Paediatr Anaesth 2014; 24: 11271131.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8. Shavit I, Kofman M, Leder M, et al. Observational pain assessment versus self-report in paediatric triage. Emerg Med J 2008; 25: 552555.

  • 9. Maciocia PM, Strachan EM, Akram AR, et al. Pain assessment in the paediatric emergency department: whose view counts? Eur J Emerg Med 2003; 10: 264267.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 10. Benito J, Monteiro BP, Lavoie AM, et al. The analgesic efficacy of intraperitoneal administration of bupivacaine in cats. J Feline Med Surg 2016; 18: 906912.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11. Luna SPL, Brondani JT. Pain assessment in cats. Animal pain website. Available at: www.animalpain.com.br/en-us/avalie-sua-habilidade.php. Accessed May 5, 2014.

    • Search Google Scholar
    • Export Citation
  • 12. Benito-de-la-Vibora J, Lascelles BD, Garcia-Fernandez P, et al. Efficacy of tolfenamic acid and meloxicam in the control of postoperative pain following ovariohysterectomy in the cat. Vet Anaesth Analg 2008; 35: 501510.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13. Lascelles BD, Cripps P, Mirchandani S, et al. Carprofen as an analgesic for postoperative pain in cats: dose titration and assessment of efficacy in comparison to pethidine hydrochloride. J Small Anim Pract 1995; 36: 535541.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 14. Steagall PV, Taylor PM, Rodrigues LC, et al. Analgesia for cats after ovariohysterectomy with either buprenorphine or carprofen alone or in combination. Vet Rec 2009; 164: 359363.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 15. Giordano T, Steagall PV, Ferreira TH, et al. Postoperative analgesic effects of intravenous, intramuscular, subcutaneous or oral transmucosal buprenorphine administered to cats undergoing ovariohysterectomy. Vet Anaesth Analg 2010; 37: 357366.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 16. Altman DG. Some common problems in medical research. In: Practical statistics for medical research. London: Chapman and Hall, 1991; 404408.

    • Search Google Scholar
    • Export Citation
  • 17. Carpenter RE, Wilson DV, Evans AT. Evaluation of intraperitoneal and incisional lidocaine or bupivacaine for analgesia following ovariohysterectomy in the dog. Vet Anaesth Analg 2004; 31: 4652.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 18. Campagnol D, Teixeira-Neto FJ, Monteiro ER, et al. Effect of intraperitoneal or incisional bupivacaine on pain and the analgesic requirement after ovariohysterectomy in dogs. Vet Anaesth Analg 2012; 39: 426430.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 19. Mathews K, Kronen PW, Lascelles D, et al. Guidelines for recognition, assessment and treatment of pain. J Small Anim Pract 2014; 55: E10E68.

  • 20. Firth AM, Haldane SL. Development of a scale to evaluate postoperative pain in dogs. J Am Vet Med Assoc 1999; 214: 651659.

  • 21. Reid J. Nolan A. A comparison of the postoperative analgesic and sedative effects of flunixin and papaveretum in the dog. J Small Anim Pract 1991; 32: 603608.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 22. Nolan A, Reid J. Comparison of the postoperative analgesic and sedative effects of carprofen and papaveretum in the dog. Vet Rec 1993; 133: 240242.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 23. Holton L, Reid J, Scott EM, et al. Development of a behaviour-based scale to measure acute pain in dogs. Vet Rec 2001; 148: 525531.

  • 24. Holton LL, Scott EM, Nolan AM, et al. Comparison of three methods used for assessment of pain in dogs. J Am Vet Med Assoc 1998; 212: 6166.

    • Search Google Scholar
    • Export Citation
  • 25. Moran CE, Hofmeister EH. Prevalence of pain in a university veterinary intensive care unit. J Vet Emerg Crit Care San Antonio 2013; 23: 2936.

  • 26. Criste A. Do nurse anesthetists demonstrate gender bias in treating pain? A national survey using a standardized pain model. AANA J 2003; 71: 206209.

    • Search Google Scholar
    • Export Citation
  • 27. Criste A. AANA journal course. Update for nurse anesthetists. Gender and pain. AANA J 2002; 70: 475480.

  • 28. Hoffmann DE, Tarzian AJ. The girl who cried pain: a bias against women in the treatment of pain. J Law Med Ethics 2001; 29: 1327.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 29. Nevin K. Influence of sex on pain assessment and management. Ann Emerg Med 1996; 27: 424426.

  • 30. Hunt JR, Knowles TG, Lascelles BD, et al. Prescription of perioperative analgesics by UK small animal veterinary surgeons in 2013. Vet Rec 2015; 176: 493.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 31. Capner CA, Lascelles BD, Waterman-Pearson AE. Current British veterinary attitudes to perioperative analgesia for dogs. Vet Rec 1999; 145: 9599.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 538 0 0
Full Text Views 1909 1609 179
PDF Downloads 641 318 32
Advertisement