Research & Efficacy

Research & Efficacy

Visually diagnosed medical tests (e.g. radiographs) are the most ordered tests in front-line medicine. As such, front-line health care professionals are faced with the task of learning the skill of interpreting these images to an expert performance level by the time they provide opinions that guide patient management decisions. However, discordant interpretations of these images between front-line physicians and expert counterparts (e.g. radiologists) is a common cause of medical error, and this can be as high as 30%.

Research & Efficacy

Our research to date shows that this learning method works – physicians at varying levels of expertise significantly increase their accuracy in image interpretation by about 15% after just an hour of practice.

ImageSim™ provides a comprehensive and evidence-based on-line education system that teaches health care professionals the interpretation of visually diagnosed medical tests. Our system integrates grounded education theory such as cognitive simulation, deliberate practice, and competency-based medicine. As such, our learning model includes sustained active practice of hundreds of cases that represents the spectrum of clinical presentations, and the learner commits to diagnosis for every case after which they receive immediate specific feedback on their interpretation so that the participant instantly learns from each case. Participants are also tasked to achieve a competency standard derived from data of practising physicians.

Research & Efficacy

Our research to date shows that this learning method works – physicians at varying levels of expertise significantly increase their accuracy in image interpretation by about 15% after just an hour of practice.

Selected Publications

  1. 1. Goertzen E, Casas M, Barrett E, Perschbauer S, Boutis K. Interactive Computer Assisted Learning as an Educational Method for Learning Pediatric Interproximal Dental Caries Identification. Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology. 2023. In press.
  2. 2. Harel-Sterling M, Kwan CK, Pirie J, Tessaro M, Cho DD, Coblentz A, Halabi M, Cohen E, Nield LE, Pusic M, Boutis K. Competency Standard Derivation for Point-of-Care Ultrasound Image Interpretation for Emergency Physicians. Annals of EM.2023: S0196-0644(22)01253-7.
  3. 3. Bregman S, Thau E, Pusic M, Perez M, Boutis K. Pediatric Chest Radiograph Interpretation: A Case for Implementing Competency Assessments Among Practicing Emergency Physicians. JCHCP. EPub Dec 21, 2022. DOI: 10.1097.
  4. 4. Li W, Stimec J, Camp M, Pusic M, Herman J, Boutis K. Pediatric Musculoskeletal Radiographs: Anatomy and Fractures Prone to Diagnostic Error Among Emergency Physicians. JEM. 2022 Apr;62(4):524-533.
  5. 5. Lee MS, Pusic M, Pecaric M, Camp M, Stimec J, Carrier B, Dixon A, Herman J, Boutis K. A Target Population Derived Method for Developing a Competency Standard in Radiograph Interpretation. Med Teach and Learning. 2022 Apr-May;34(2):167-177.
  6. 6. Thau E, Perez M, Pusic M, Pecaric M, Rizutti D, Boutis K. Image Interpretation: Learning Analytics Informed Education Opportunities. AEM Educ and Training. 2021 Apr 1;5(2):e10592.
  7. 7. Kwan C, Weerdenburg K, Pusic M, Chen A, Rempell R, Hermann J, Boutis K. Learning Pediatric Point-of-Care Ultrasound: How Many Cases Does Mastery of Image Interpretation Take? Pediatric Emergency Care. In press.
  8. 8. Campos S, Pusic MV, Smith T, Legano L, Brown J, Pecaric M, Boutis K. Prepubertal Female Genital Examination: Evidence Informed Learning Opportunities. Journal of Pediatric and Adolescent Gynecology. 2021 Apr;34(2):117-123.
  9. 9. Adelgais K, Pusic MV, Abdoo D, Caffrey S, Snyder K, Boutis K. Child Abuse Recognition Training using Deliberate Practice for Prehospital Providers. Prehospital Care. Epub. November 11, 2020.
  10. 10. Boutis K, Pecaric M, Stimec J, Carrier B, Chan J, Pusic M. Radiograph Interpretation and Skill Retention: A Multicentre Randomized Control Trial. Med Teach. 2019 May 2:1-9.
  11. 11. Lee M, Pusic M, Carriere B, Dixon A, Pecaric M, Stimec J, Boutis K. Building Competency in Pediatric Musculoskeletal Radiograph Interpretation: A Multicentre Prospective Cohort Study. Acad Med – Education and Training. 2019 Mar 12;3(3):269-279.
  12. 12. Kwan C, Pusic M, Pecaric M, Weerdenburg K, Tesssaro M, Boutis K. The Variable Journey in Learning to Interpret Pediatric Point-of-care Ultrasound Images: A Multicenter Prospective Cohort Study. AEM Educ Train. 2019 Jul 30;4(2):111-122.
  13. 13. Yoon P, Boutis K, Pecaric M, Fefferman NR, Ericsson A, Pusic M. A Think-Aloud Study to Inform the Design of Radiograph Interpretation Practice. AHSE. 2020. 25(4): 877-903.
  14. 14. Pusic M, Boutis K, McGaghie W. Role of Scientific Theory in Simulation Education Research. Simulation in Healthcare. 2018 Jan 24 Epub ahead of print.
  15. 15. Pusic M, Boutis K, Pecaric M, Savenkov O, Beckstead JW, Jabour M. A primer on the statistical modelling of learning curves in health professions education. Adv Health Sci Educ Theory Pract. 2017 Aug;22(3):741-759
  16. 16. Pecaric M, Boutis K, Beckstead J, Pusic M. A Big Data Learning Analytics Approach to Process-Level Feedback in Cognitive Simulations. Acad Med. 2017 Feb;92(2):175-184.
  17. 17. Beckstead J, Boutis K, Pecaric M, Pusic M. Sequential Dependencies in Categorical Judgments of Radiographic Images. Adv Health Sci Educ Theory Pract. 2017 Mar;22(1):197-207
  18. 18. Pusic M, Chiaramonte R, Pecaric M, Boutis K. Accuracy of self-monitoring during learning of radiograph interpretation. Med Educ. 2015 Aug;49(8):838-46.
  19. 19. Pusic M, Boutis K, Hatala R, Cook D. Learning curves: A Primer for Health Professions Education. Acad Med. 2014. Aug;90(8):1034-42
  20. 20. Beckstead J, Boutis K, Pecaric M, Pusic M. Sequential Dependencies in Categorical Judgments of Radiographic Images. AHSE. Jun 8. 2016.
  21. 21. Boutis K, Cano S, Pecaric M, et al. Interpretation Difficulty of Normal versus Abnormal Radiographs Using a Paediatric Example. CMEJ. 2016 Mar 31;7(1):e68-77.
  22. 22. Pusic M, Chiaramonte R, Pecaric M, Boutis K. A Simple Method for Improving Self-Assessment While Learning Radiograph Interpretation. Med Educ. 2015;49(8):838-46.
  23. 23. Pusic M, Boutis K, Hatala R, Cook D. Learning curves: A Primer for Health Professions Education. Acad Med. 2014;90(8):1034-1042.
  24. 24. Beckstead J, Boutis K, Pecaric M, Pusic M. The Influences of Sequential Effects in Randomized Online Learning Tasks. Applied Cognition. 2014; 27(5):625-632
  25. 25. Boutis K, Pecaric M, Ridley J, Andrews J, Gladding S, Pusic M. Hinting Strategies for Improving the Efficiency of Medical Student Learning of Deliberately Practiced Web-based Radiographs. Med Educ. 2013;47(9):877-887.
  26. 26. Pusic MV, Andrews JS, Kessler DO, Boutis K. Determining the optimal case mix of abnormals to normals for learning radiograph interpretation: a randomized control trial. Medical Education. 2012;46(3):289-298.
  27. 27. Pusic MV, Kessler D, Szyld D, Kalet A, Pecaric M, Boutis K. Experience Curves as an Organizing Framework for Deliberate Practice in Emergency Medicine Learning. Acad Emerg Med. 2012;19(12):1476-80.
  28. 28. Pusic M, Pecaric M, Boutis K. How Much Practice is Enough? Using Learning Curves to Assess the Deliberate Practice of Radiograph Interpretation. Acad Med. 2011;86(6):731-736.
  29. 29. Boutis K, Pecaric M, Pusic M. Using Signal Detection Theory to Model Changes In Serial Learning of Radiological Image Interpretation. Adv Health Sci Educ Theory Pract. 2010;15:647-658.
  30. 30. Boutis K, Pecaric M, Pusic M. Selecting Radiographs For Teaching The Interpretation of X-Rays Based On Ratings by the Target Population. Med Educ. 2009;43(5):434-441.