Abbreviations
| DIMT | Digital interactive multimedia tutorial |
| FLASH | Fast localized abdominal ultrasonography of horses |
Introduction
High-resolution ultrasound equipment is becoming increasingly less expensive and easier to use, allowing more general practitioners to integrate this technology into daily veterinary practice.1,2 With ultrasonography becoming a part of primary care in veterinary medicine,3 the opportunity exists to incorporate this diagnostic technique into veterinary curricula through the teaching and reinforcement of clinically applicable information. However, a paucity of data exists regarding the most effective means of training veterinary students in ultrasonography, and the debate continues in human medical education on best practices for training undergraduate medical students. Some programs provide instruction and practice in ultrasonography to help reinforce coursework in anatomy and physiology,4,5 and others provide practice in ultrasonography to improve student confidence in medical procedures.6 Indeed, undergraduate medical students with limited training in point-of-care ultrasonography were able to more accurately assess liver size when compared with experienced physicians using physical examination alone.7 Despite the agreement that training in ultrasonography belongs in medical education, its integration across schools is highly variable.8 We believe this is true in veterinary medical education as well.
Whereas the primary means of instruction in diagnostic imaging in veterinary curricula has a didactic focus, the use of instructional methods that incorporate active learning components is advocated in educational science to improve student learning.9 We believe that today's students challenge previous pedagogical paradigms and demand that academicians provide more creative mechanisms to support student learning. Interactive multimedia materials coupled with hands-on activities could be an ideal combination to deliver educational content in an effective and sustainable manner.
The aim of the study reported here was to determine the effectiveness of a DIMT versus independent study or lecture for preparing third-year veterinary students at a North American veterinary school to perform ultrasonography of the palmar metacarpal area and abdomen in live horses. We hypothesized that students educated on how to perform ultrasonography in horses by means of the DIMT would score higher on written and practical tests than would students educated by textbook chapter review or traditional lecture.
Materials and Methods
Participants
The study protocol was approved by the Human Research Protection Program and Institutional Animal Care and Use Committee at Texas A&M University. Third-year veterinary students enrolled in a core course on diagnostic imaging and interpretation were invited to participate. An a priori sample size calculation indicated that a sample size of 7 participants/education group would allow detection of a difference of 2 points in the written and practical tests if the SD of the mean for each group was 1.5.
Study design
Two experiments were conducted as randomized trials. Each experiment included 45 minutes of instructional content, a 15-minute written test, and a 20-minute practical test. The first experiment was designed to compare a DIMT and an independent textbook chapter reading as methods for preparing students to perform ultrasonography of the palmar metacarpal area in horses.10,11 The second experiment was designed to compare the DIMT approach and a traditional lecture as methods for preparing students to perform FLASH.12
For experiment 1, 18 students volunteered and were randomly assigned to 2 groups: DIMT (n = 9) or independent textbook chapter reading (ie, textbook group; 9). Students in the DIMT group watched a 45-minute interactive, narrated tutorial on knobology (ie, basic principles and fundamental functions necessary to optimize image quality) to gain familiarity with use of an ultrasound machine and how to perform ultrasonography of the palmar metacarpal area in horses. Students in the textbook group were allotted 45 minutes to review a textbook chapter with highlighted information analogous to the content in the DIMT.
For experiment 2, 24 students volunteered and were randomly assigned to 2 groups: DIMT (n = 12) or traditional lecture (12). Eleven of these students had recently participated in an elective course in equine ultrasonography that included a practical component. These 11 students were randomly assigned to the groups (6 to the DIMT group and 5 to the traditional lecture group). Students in the DIMT group watched a 45-minute DIMT on FLASH, and those in the traditional lecture group received a 45-minute lecture to provide information analogous to the content in the DIMT.
Following the learning component of both experiments, a 15-minute written test was administered (Supplementary Appendix S1, available at: avmajournals.avma.org/doi/suppl/10.2460/javma.258.2.165). The written test for experiment 1 included questions regarding the recognition of normal ultrasonographic distal limb anatomy, the adjustments necessary to improve image quality for a given image, and the identification of soft tissue structures with abnormalities. The written test for experiment 2 included questions related to the identification of normal ultrasonographic abdominal anatomy, the identification and diagnosis of abnormal anatomy, and the clinical implications of abnormal findings. Then, for the live animal portion of the study, students were provided with a 5-minute orientation to the ultrasound equipment and allowed 20 minutes to practice independently and complete the practical test.
The practical test (Supplementary Appendix S2, available at: avmajournals.avma.org/doi/suppl/10.2460/javma.258.2.165) included use of an ultrasound machinea to capture predetermined images, which students stored digitally and which were identified only by the student's randomly assigned number. Students then received 30 minutes of individual feedback on their ultrasonography skills from experienced clinicians.
DIMTs
In both experiments, the narrated DIMTs consisted of a slide presentationb designed by the authors, who included a board-certified equine internist with advanced training in ultrasonography (CN), a board-certified radiologist (LJG), and a board-certified equine surgeon (AEW) for the palmar metacarpal area or board-certified equine internist (MCC) for FLASH. Each DIMT contained a combination of slides with digital images, video clips, and a small amount of text. The narrated component of the tutorials encouraged student interaction through the use of graphics and embedded activities (eg, video examples, case vignettes, and question-and-answer review slides).
Textbook chapter
For experiment 1, the textbook content10,11 was assigned to provide information comparable to that in the DIMT, including similar anatomic figures and graphics outlined in the text and in the DIMT. To help students prioritize their independent study time, chapter content analogous to the DIMT content was outlined with red boxes.
Lecture
For experiment 2, a 45-minute lecture (the same duration as the recorded DIMT) was presented by the same author (CN) who also developed the DIMT for experiment 2. This lecture provided content similar to that presented in the DIMT for FLASH and was delivered via lecture-style seminar format without interactive exercises.
Tests
The written tests were provided in paper and digital formats to students in each instruction group for each experiment. Test questions were scored with a rubric on a scale of 0 to 2 (0 = incorrect, 1 = partially correct, and 2 = fully correct) by an investigator who was blinded to group assignment. Some questions had > 1 section. For experiments 1 and 2, the maximum possible score was 14 and 10, respectively.
Images obtained by students in the practical test were later blindly reviewed and scored by an author (CN), who evaluated whether the correct anatomic location and use of adequate sonographic technique (ie, depth, gain, frequency, and focal zone) had been chosen and scored the images accordingly on a scale of 0 to 2, as for the written test. For experiments 1 and 2, the maximum possible score was 10 and 14, respectively.
Postexperiment survey
Twenty-four hours after completing their experiment, students were given access to all study materials and asked to complete a brief survey regarding their experience using the assigned instructional method and its impact on their ability to perform ultrasonography in the live-animal practical session.
Statistical analysis
For descriptive purposes, data plots and summary statistics were generated with the aid of statistical software.c Results of the written test and practical test for each experiment were analyzed separately. Normality of data distribution was assessed with the Shapiro-Wilk method. Normally distributed data were compared between instruction groups with the Student t test. Nonnormally distributed data were compared between instruction groups with the Wilcoxon signed rank test. The χ2 test was used to test differences between instruction groups in proportions of students using a standoff pad during the practical test of experiment 1.
As a method of evaluating qualitative data from the postexperiment survey, grounded theory methods13 were used to develop a theoretical model by analyzing the free-text comments from the sixth and final question in the survey (“Please use the space below to provide any other comments you feel could be useful to design future ultrasound courses for students”). The free-text comments were coded line by line. These open codes identified distinct concepts and relationships within the raw data, which were then organized and grouped by their shared properties into 4 conceptual categories. These 4 categories may serve as the foundation of a theoretical model that can be used for designing future ultrasonography courses on the basis of participant feedback.
Results
Written and practical tests
Test scores were normally distributed, except scores for the practical test in experiment 2. For experiment 1, students in the DIMT group scored higher on the practical test than did students in the independent book chapter reading group (mean difference in scores, 1.4; P = 0.04; Table 1). No difference was identified between groups in the written test scores in experiment 1 or 2 or in practical test scores in experiment 2. In experiment 1, despite similar information being available to both instruction groups, a significantly (P = 0.01) higher proportion of students (9/9) in the DIMT group used a standoff pad to obtain images of the palmar metacarpal area than did students in the textbook group (2/9).
Summary statistics for scores on practical and written tests completed by third-year veterinary students after 45 minutes of instruction in ultrasonography by means of a DIMT (n = 9 for experiment 1 and 12 for experiment 2), independent textbook chapter reading (9), or traditional lecture (12).
| Test, by experiment | Instructional method | Mean | SD | Median | Range | Maximum possible score |
|---|---|---|---|---|---|---|
| Experiment 1 | ||||||
| Written | DIMT | 8.6 | 2.1 | 9.2 | 7.1–10.7 | 14 |
| Textbook | 8.6 | 0.9 | 8.9 | 6.6–9.9 | 14 | |
| Practical | DIMT | 7.6* | 1.4 | 8.1 | 5.0–9.3 | 10 |
| Textbook | 6.2* | 1.4 | 6.0 | 2.9–8.1 | 10 | |
| Experiment 2 | ||||||
| Written | DIMT | 8.2 | 1.2 | 7.6 | 5.6–9.7 | 10 |
| Lecture | 7.6 | 1.4 | 8.2 | 4.8–9.7 | 10 | |
| Practical | DIMT | 10.1 | 3.7 | 10.0 | 3.0–14.0 | 14 |
| Lecture | 10.6 | 3.4 | 11.0 | 5.0–14.0 | 14 |
Indicated values differ significantly (P = 0.04) from each other.
Postexperiment survey
Results of the postexperiment survey were summarized (Supplementary Table S1, available at: avmajournals.avma.org/doi/suppl/10.2460/javma.258.2.165). Four major categories were identified among free-text comments. Responses in category 1 suggested the effectiveness of the practical portion of the study involving a live horse was dependent on the students’ previous experience with ultrasonography. Results indicated that a lack of prior exposure, hands-on training, or knobology training hindered learning and left students with only a superficial understanding of how to perform ultrasonography in horses. Additionally, students felt image quality was better when they had previous practical experience. For category 2, students defined various mechanisms that improved their learning, including having more time to learn and practice, having opportunities for trial and error in which they worked independently prior to receiving feedback from an expert, having one-on-one interaction with an expert, and understanding the clinical relevance of the material. For category 3, instructional methods that enhanced student ultrasonography training were identified. These included real-time instruction from an expert and individual feedback. Students also mentioned primer materials such as textbook chapters, a narrated digital presentation,b and a prelaboratory lecture as having a positive impact on their performance. For category 4, students reported that participation in the study increased their comfort with equine ultrasonography (ie, performing the examination and identifying pertinent anatomic structures), improved their ability to obtain quality ultrasound images, and strengthened their learning.
Discussion
In the present study, third-year veterinary students using a DIMT as a method of independent learning scored higher on a practical test of ultrasonography skills than did students reading analogous textbook content. It has been suggested that the use of digital technology (ie, e-learning), such as the DIMTs used in this study, may be as effective as other instructional methods,14,15 although additional research is needed to determine the effectiveness of e-learning techniques for specific applications or uses in clinical practice.16 Student comments captured in the postexperiment survey reflected their preference for practicing techniques on live animals above other instructional methods; however, the impact of DIMTs on learning was demonstrated through relevant measurable outcomes (ie, superior quality of ultrasound images obtained by students in the DIMT group vs those in the textbook group in experiment 1).
In both experiments, nearly all students reported that the most impactful portion was when they could practice on a live horse during the 20 minutes allowed to complete the practical test. We question whether DIMTs would have ranked higher as the most impactful instructional method if the “practical hands-on laboratory” session had not been included as a choice in the provided list of instructional methods from which students could choose. Practical test scores were higher for students who learned how to perform ultrasonography by completing a DIMT than for those who learned about it by reading a textbook chapter, but this advantage was no longer evident when the DIMT was compared with learning via traditional lecture. These findings, combined with student comments regarding the impact of real-time instructor feedback on training, suggested that regardless of the instructional method, opportunities to practice the techniques learned on live animals would be important and an essential component of ultrasonography training for veterinary students.
When new educational technologies are developed and implemented to enhance and integrate learning of multiple critical components of medical or veterinary education such as anatomy, physiology, physical examination, or mechanisms of disease or therapeutics,17 research is necessary to assess the technologies’ impact on education. Some resistance exists regarding untested, unregulated proliferation of innovations in medical education and their unintended consequences.18 A recent study19 showed no difference in performance between medical students using a mobile application and those using textbooks to learn how to perform a point-of-care ultrasound technique analogous to FLASH, although textbook-guided learning was more cost-effective.
Qualitative results of the study reported here suggested that the practical component and subsequent student interaction with experienced ultrasonographers were key elements of the hands-on training. In human medicine, expert-guided training of students in cardiac ultrasonography has been found to provide measurable outcomes superior to those of self-directed training, and simulation training alone has not been found to be equivalent to practical small group instruction under the supervision of experienced ultrasonographers.19 Similar to other studies,19,20 the present study revealed no difference in written test scores between veterinary students exposed to a lecture approach and those exposed to an e-learning approach, supporting the importance of defining relevant outcome measurements to confirm the impact and positive effects of various instructional methods.
An unexpected finding of the present study was the observation that all 9 veterinary students who learned how to perform ultrasonography in horses by means of DIMT used a standoff pad for imaging the distal forelimb region in a live horse, whereas only 2 of the 9 students who learned by independent textbook chapter reading did so. The use of standoff pads is beneficial when imaging this area in horses and was explained in the textbook chapter and the multimedia materials with similar emphasis. In addition, a standoff pad was made equally available for all students during the practical tests. Although its purpose was not reemphasized for either experimental group, the standoff pad was placed on top of the ultrasound equipment keyboard as an implicit invitation for use. Use of the standoff pad by all students in the DIMT group during the practical test was likely an unintended consequence of the use of images and videos in the DIMT, which showed operators using a standoff pad when imaging the distal limb region.
A limitation of the study reported here was that only short-term learning was assessed. Substantiation of the impact of multimedia instruction followed by hands-on practice could be achieved by assessment of long-term skills and knowledge retention.21 A recent study22 showed that first-year veterinary students engaged in independent active learning had improved short- and long-term knowledge retention and performance related to ultrasound knobology and image quality recognition, compared with students engaged in other content-delivery methods, such as in-person instruction or online module instruction. Another limitation of the present study was that the lecture and interactive materials were designed by the same investigator, whereas the textbook chapters were not. However, the textbook used was chosen as a basis for development of the digital interactive media to ensure that analogous material was included in the DIMT.
The learning process is complex and challenging to measure, even when a combination of objective variables and subjective surveys is used, as in the present study. Further comparisons and assessment of the long-term influence of the instructional methods evaluated here and their integration with and contribution to the rest of the veterinary curriculum are needed to confirm the most effective method for teaching ultrasonography in veterinary medical education. Third-year veterinary students demonstrated the ability to obtain basic musculoskeletal ultrasound images of higher quality after reviewing a DIMT than after reading analogous information in a textbook. As the use of ultrasonography continues to increase in veterinary practice, educational mechanisms should continue to be developed to help students to learn and train optimally. A combination of instructional methods followed by hands-on training with instructor feedback should be considered when designing ultrasonography training experiences for veterinary students.
Acknowledgments
No third-party funding or support was received in connection with this study or the writing or publication of the manuscript. The authors declare that there were no conflicts of interest.
The authors thank Dr. Maria Macik for her contributions in qualitative data analysis and Erin Lester for assistance with organization of the experiments.
Footnotes
GE Logiq e, GE Healthcare, Chicago, Ill.
PowerPoint, Microsoft Corp, Redmond, Wash.
JMP, version 14, SAS Institute, Cary, NC.
References
- 1. ↑
So S, Patel RM, Orebaugh SL. Ultrasound imaging in medical student education: impact on learning anatomy and physical diagnosis. Anat Sci Educ 2017;10:176–189.
- 2. ↑
Johnson V. Diagnostic imaging: reflecting on the past and looking to the future. Vet Rec 2013;172:546–551.
- 3. ↑
Booth NJ, Morley SJ, Ewers RS. Use of radiography in small animal practice in the UK and Republic of Ireland in 2013. Vet Rec 2018;182:225.
- 4. ↑
Griksaitis MJ, Scott MP, Finn GM. Twelve tips for teaching with ultrasound in the undergraduate curriculum. Med Teach 2014;36:19–24.
- 5. ↑
Mircea P-A, Badea R, Fodor D, et al. Using ultrasonography as a teaching support tool in undergraduate medical education—time to reach a decision. Med Ultrason 2012;14:211–216.
- 6. ↑
Goolsby CA, Goodwin TL, Vest RM. Hybrid simulation improves medical student procedural confidence during EM clerkship. Mil Med 2014;179:1223–1227.
- 7. ↑
Mouratev G, Howe D, Hoppmann R, et al. Teaching medical students ultrasound to measure liver size: comparison with experienced clinicians using physical examination alone. Teach Learn Med 2013;25:84–88.
- 8. ↑
Bahner DP, Goldman E, Way D, et al. The state of ultrasound education in US medical schools: results of a national survey. Acad Med 2014;89:1681–1686.
- 9. ↑
Sinnayah P, Rathner JA, Loton D, et al. A combination of active learning strategies improves student academic outcomes in first-year paramedic bioscience. Adv Physiol Educ 2019;43:233–240.
- 10. ↑
Palgrave K, Kidd J. Introduction. In: Kidd J, Lu K, Frazer M, eds. Atlas of equine ultrasonography. New York: John Wiley & Sons, 2014;6–11.
- 11. ↑
Smith R, Cauvin E. Ultrasonography of the metacarpus and metatarsus. In: Kidd J, Lu K, Frazer M, eds. Atlas of equine ultrasonography. New York: John Wiley & Sons, 2014;73–86.
- 12. ↑
Busoni V, De Busscher V, Lopez D, et al. Evaluation of a protocol for fast localised abdominal sonography of horses (FLASH) admitted for colic. Vet J 2011;188:77–82.
- 13. ↑
Strauss A, Corbin J. Basics of qualitative research: techniques and procedures for developing grounded theory. 4th ed. Thousand Oaks, Calif: Sage Publications Inc, 2015.
- 14. ↑
Farah CS, Maybury T. Implementing digital technology to enhance student learning of pathology. Eur J Dent Educ 2009;13:172–178.
- 15. ↑
Maertens H, Madani A, Landry T, et al. Systematic review of e-learning for surgical training. Br J Surg 2016;103:1428–1437.
- 16. ↑
Nilsson PM, Todsen T, Subhi Y, et al. Cost-effectiveness of mobile app-guided training in extended focused assessment with sonography for trauma (eFAST): a randomized trial. Ultraschall Med 2017;38:642–647.
- 17. ↑
Solomon SD, Saldana F. Point-of-care ultrasound in medical education—stop listening and look. N Engl J Med 2014;370:1083–1085.
- 18. ↑
Feilchenfeld Z, Kuper A, Whitehead C. Stethoscope of the 21st century: dominant discourses of ultrasound in medical education. Med Educ 2018;52:1271–1287.
- 19. ↑
Cawthorn TR, Nickel C, O'Reilly M, et al. Development and evaluation of methodologies for teaching focused cardiac ultrasound skills to medical students. J Am Soc Echocardiogr 2014;27:302–309.
- 20. ↑
Soon AW, Toney AG, Stidham T, et al. Teaching point-of-care lung ultrasound to novice pediatric learners: web-based e-learning versus traditional classroom didactic. Pediatr Emerg Care 2020;36:317–321.
- 21. ↑
Sites BD, Gallagher JD, Craver J, et al. The learning curve associated with a simulated ultrasound-guided interventional task by inexperienced anesthesia residents. Reg Anesth Pain Med 2004;29:544–548.
- 22. ↑
Scallan EM, Voges AK, Chaney KP, et al. The effects of content delivery methods on ultrasound knobology and image quality recognition training in first-year veterinary students [published online ahead of print Nov 15, 2019]. J Vet Med Educ doi: 10.3138/jvme.2019-0014.
