TY - JOUR

T1 - 3-D-Printed Models for Temporal Bone Training

T2 - A Validity Study

AU - Frithioff, Andreas

AU - Frendø, Martin

AU - Weiss, Kenneth

AU - Foghsgaard, Søren

AU - Mikkelsen, Peter Trier

AU - Frederiksen, Thomas Winther

AU - Pedersen, David Bue

AU - Sørensen, Mads Sølvsten

AU - Andersen, Steven Arild Wuyts

N1 - Publisher Copyright: © Wolters Kluwer Health, Inc. All rights reserved.

PY - 2023

Y1 - 2023

N2 - Objective 3-D printing offers convenient and low-cost mastoidectomy training; nonetheless, training benefits using 3-D-printed temporal bones remain largely unexplored. In this study, we have collected validity evidence for a low-cost, 3-D-printed temporal bone for mastoidectomy training and established a credible pass/fail score for performance on the model. Study design A prospective educational study gathering validity evidence using Messick's validity framework. Setting Seven Danish otorhinolaryngology training institutions. Participants Eighteen otorhinolaryngology residents (novices) and 11 experienced otosurgeons (experts). Intervention Residents and experienced otosurgeons each performed two to three anatomical mastoidectomies on a low-cost, 3-D-printed temporal bone model produced in-house. After drilling, mastoidectomy performances were rated by three blinded experts using a 25-item modified Welling scale (WS). Main outcome measure Validity evidence using Messick's framework including reliability assessment applying both classical test theory and Generalizability theory. Results Novices achieved a mean score of 13.9 points; experienced otosurgeons achieved 23.2 points. Using the contrasting groups method, we established a 21/25-point pass/fail level. The Generalizability coefficient was 0.91, and 75% of the score variance was attributable to participant performance, indicating a high level of assessment reliability. Subsequent D studies revealed that two raters rating one performance or one rater rating two performances were sufficiently reliable for high-stakes assessment. Conclusion Validity evidence supports using a low-cost, 3-D-printed model for mastoidectomy training. The model can be printed in-house using consumer-grade 3-D printers and serves as an additional training tool in the temporal bone curriculum. For competency-based training, we established a cut-off score of 21 of 25 WS points using the contrasting groups method.

AB - Objective 3-D printing offers convenient and low-cost mastoidectomy training; nonetheless, training benefits using 3-D-printed temporal bones remain largely unexplored. In this study, we have collected validity evidence for a low-cost, 3-D-printed temporal bone for mastoidectomy training and established a credible pass/fail score for performance on the model. Study design A prospective educational study gathering validity evidence using Messick's validity framework. Setting Seven Danish otorhinolaryngology training institutions. Participants Eighteen otorhinolaryngology residents (novices) and 11 experienced otosurgeons (experts). Intervention Residents and experienced otosurgeons each performed two to three anatomical mastoidectomies on a low-cost, 3-D-printed temporal bone model produced in-house. After drilling, mastoidectomy performances were rated by three blinded experts using a 25-item modified Welling scale (WS). Main outcome measure Validity evidence using Messick's framework including reliability assessment applying both classical test theory and Generalizability theory. Results Novices achieved a mean score of 13.9 points; experienced otosurgeons achieved 23.2 points. Using the contrasting groups method, we established a 21/25-point pass/fail level. The Generalizability coefficient was 0.91, and 75% of the score variance was attributable to participant performance, indicating a high level of assessment reliability. Subsequent D studies revealed that two raters rating one performance or one rater rating two performances were sufficiently reliable for high-stakes assessment. Conclusion Validity evidence supports using a low-cost, 3-D-printed model for mastoidectomy training. The model can be printed in-house using consumer-grade 3-D printers and serves as an additional training tool in the temporal bone curriculum. For competency-based training, we established a cut-off score of 21 of 25 WS points using the contrasting groups method.

KW - 3-D printing

KW - Additive manufacturing

KW - Assessment

KW - Mastoidectomy

KW - Medical education

KW - Neurotology

KW - Otology

KW - Rapid prototyping

KW - Surgical simulation

KW - Temporal bone

KW - Training

U2 - 10.1097/MAO.0000000000003936

DO - 10.1097/MAO.0000000000003936

M3 - Journal article

C2 - 37442608

AN - SCOPUS:85164843203

VL - 44

SP - e497-e503

JO - Otology & Neurotology

JF - Otology & Neurotology

SN - 1531-7129

IS - 7

ER -