Increasing completion rate of an M4 emergency medicine student end-of-shift evaluation using a mobile electronic platform and real-time completion

Matthew C Tews, Robert W. Treat, Maxwell Nanes

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Introduction: Medical students on an emergency medicine rotation are traditionally evaluated at the end of each shift with paper-based forms, and data are often missing due to forms not being turned in or completed. Because students' grades depend on these evaluations, change was needed to increase form rate of return. We analyzed a new electronic evaluation form and modified completion process to determine if it would increase the completion rate without altering how faculty scored student performance. Methods: During fall 2013, 29 faculty completed paper N=339 evaluations consisting of seven competencies for 33 students. In fall 2014, an electronic evaluation form with the same competencies was designed using an electronic platform and completed N=319 times by 27 faculty using 25 students' electronic devices. Feedback checkboxes were added to facilitate collection of common comments. Data was analyzed with IBM® SPSS® 21.0 using multi-factor analysis of variance with the students' global rating (GR) as an outcome. Inter-item reliability was determined with Cronbach alpha. Results: There was a significantly higher completion rate (p=0.001) of 98% electronic vs. 69% paper forms, lower (p=0.001) missed GR rate (1% electronic. vs 12% paper), and higher mean scores (p=0.001) for the GR with the electronic (7.0±1.1) vs. paper (6.8±1.2) form. Feedback checkboxes were completed on every form. The inter-item reliability for electronic and paper forms was each alpha=0.95. Conclusion: The use of a new electronic form and modified completion process for evaluating students at the end of shift demonstrated a higher faculty completion rate, a lower missed data rate, a higher global rating and consistent collection of common feedback. The use of the electronic form and the process for obtaining the information made our end-of-shift evaluation process for students more reliable and provided more accurate, up-to-date information for student feedback and when determining student grades.

Original languageEnglish (US)
Pages (from-to)478-483
Number of pages6
JournalWestern Journal of Emergency Medicine
Volume17
Issue number4
DOIs
StatePublished - Jan 1 2016
Externally publishedYes

Fingerprint

Emergency Medicine
Students
Medical Students
Statistical Factor Analysis
Analysis of Variance
Equipment and Supplies

ASJC Scopus subject areas

  • Emergency Medicine

Cite this

Increasing completion rate of an M4 emergency medicine student end-of-shift evaluation using a mobile electronic platform and real-time completion. / Tews, Matthew C; Treat, Robert W.; Nanes, Maxwell.

In: Western Journal of Emergency Medicine, Vol. 17, No. 4, 01.01.2016, p. 478-483.

Research output: Contribution to journalArticle

@article{7a37045ac45744e9a44c9d23ebd43af7,
title = "Increasing completion rate of an M4 emergency medicine student end-of-shift evaluation using a mobile electronic platform and real-time completion",
abstract = "Introduction: Medical students on an emergency medicine rotation are traditionally evaluated at the end of each shift with paper-based forms, and data are often missing due to forms not being turned in or completed. Because students' grades depend on these evaluations, change was needed to increase form rate of return. We analyzed a new electronic evaluation form and modified completion process to determine if it would increase the completion rate without altering how faculty scored student performance. Methods: During fall 2013, 29 faculty completed paper N=339 evaluations consisting of seven competencies for 33 students. In fall 2014, an electronic evaluation form with the same competencies was designed using an electronic platform and completed N=319 times by 27 faculty using 25 students' electronic devices. Feedback checkboxes were added to facilitate collection of common comments. Data was analyzed with IBM{\circledR} SPSS{\circledR} 21.0 using multi-factor analysis of variance with the students' global rating (GR) as an outcome. Inter-item reliability was determined with Cronbach alpha. Results: There was a significantly higher completion rate (p=0.001) of 98{\%} electronic vs. 69{\%} paper forms, lower (p=0.001) missed GR rate (1{\%} electronic. vs 12{\%} paper), and higher mean scores (p=0.001) for the GR with the electronic (7.0±1.1) vs. paper (6.8±1.2) form. Feedback checkboxes were completed on every form. The inter-item reliability for electronic and paper forms was each alpha=0.95. Conclusion: The use of a new electronic form and modified completion process for evaluating students at the end of shift demonstrated a higher faculty completion rate, a lower missed data rate, a higher global rating and consistent collection of common feedback. The use of the electronic form and the process for obtaining the information made our end-of-shift evaluation process for students more reliable and provided more accurate, up-to-date information for student feedback and when determining student grades.",
author = "Tews, {Matthew C} and Treat, {Robert W.} and Maxwell Nanes",
year = "2016",
month = "1",
day = "1",
doi = "10.5811/westjem.2016.5.29384",
language = "English (US)",
volume = "17",
pages = "478--483",
journal = "Western Journal of Emergency Medicine",
issn = "1936-900X",
publisher = "University of California",
number = "4",

}

TY - JOUR

T1 - Increasing completion rate of an M4 emergency medicine student end-of-shift evaluation using a mobile electronic platform and real-time completion

AU - Tews, Matthew C

AU - Treat, Robert W.

AU - Nanes, Maxwell

PY - 2016/1/1

Y1 - 2016/1/1

N2 - Introduction: Medical students on an emergency medicine rotation are traditionally evaluated at the end of each shift with paper-based forms, and data are often missing due to forms not being turned in or completed. Because students' grades depend on these evaluations, change was needed to increase form rate of return. We analyzed a new electronic evaluation form and modified completion process to determine if it would increase the completion rate without altering how faculty scored student performance. Methods: During fall 2013, 29 faculty completed paper N=339 evaluations consisting of seven competencies for 33 students. In fall 2014, an electronic evaluation form with the same competencies was designed using an electronic platform and completed N=319 times by 27 faculty using 25 students' electronic devices. Feedback checkboxes were added to facilitate collection of common comments. Data was analyzed with IBM® SPSS® 21.0 using multi-factor analysis of variance with the students' global rating (GR) as an outcome. Inter-item reliability was determined with Cronbach alpha. Results: There was a significantly higher completion rate (p=0.001) of 98% electronic vs. 69% paper forms, lower (p=0.001) missed GR rate (1% electronic. vs 12% paper), and higher mean scores (p=0.001) for the GR with the electronic (7.0±1.1) vs. paper (6.8±1.2) form. Feedback checkboxes were completed on every form. The inter-item reliability for electronic and paper forms was each alpha=0.95. Conclusion: The use of a new electronic form and modified completion process for evaluating students at the end of shift demonstrated a higher faculty completion rate, a lower missed data rate, a higher global rating and consistent collection of common feedback. The use of the electronic form and the process for obtaining the information made our end-of-shift evaluation process for students more reliable and provided more accurate, up-to-date information for student feedback and when determining student grades.

AB - Introduction: Medical students on an emergency medicine rotation are traditionally evaluated at the end of each shift with paper-based forms, and data are often missing due to forms not being turned in or completed. Because students' grades depend on these evaluations, change was needed to increase form rate of return. We analyzed a new electronic evaluation form and modified completion process to determine if it would increase the completion rate without altering how faculty scored student performance. Methods: During fall 2013, 29 faculty completed paper N=339 evaluations consisting of seven competencies for 33 students. In fall 2014, an electronic evaluation form with the same competencies was designed using an electronic platform and completed N=319 times by 27 faculty using 25 students' electronic devices. Feedback checkboxes were added to facilitate collection of common comments. Data was analyzed with IBM® SPSS® 21.0 using multi-factor analysis of variance with the students' global rating (GR) as an outcome. Inter-item reliability was determined with Cronbach alpha. Results: There was a significantly higher completion rate (p=0.001) of 98% electronic vs. 69% paper forms, lower (p=0.001) missed GR rate (1% electronic. vs 12% paper), and higher mean scores (p=0.001) for the GR with the electronic (7.0±1.1) vs. paper (6.8±1.2) form. Feedback checkboxes were completed on every form. The inter-item reliability for electronic and paper forms was each alpha=0.95. Conclusion: The use of a new electronic form and modified completion process for evaluating students at the end of shift demonstrated a higher faculty completion rate, a lower missed data rate, a higher global rating and consistent collection of common feedback. The use of the electronic form and the process for obtaining the information made our end-of-shift evaluation process for students more reliable and provided more accurate, up-to-date information for student feedback and when determining student grades.

UR - http://www.scopus.com/inward/record.url?scp=84978045025&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84978045025&partnerID=8YFLogxK

U2 - 10.5811/westjem.2016.5.29384

DO - 10.5811/westjem.2016.5.29384

M3 - Article

VL - 17

SP - 478

EP - 483

JO - Western Journal of Emergency Medicine

JF - Western Journal of Emergency Medicine

SN - 1936-900X

IS - 4

ER -