P-26 Item analysis of multiple choice and extended matching questions at the Faculty of Medical Sciences, the University of the West Indies
Author(s):
A Kumar , C George , MH Campbell , K Krishnamurthy , V Singh , S Motilal , S Sakhamuri , T Thompson , C Sinquee-Brown , PM Lashley , C Bodkyn, C Twort , B Sa , MAA Majumder
Type Of Study:
- Quantitative
- Methodological Studies
Country(ies) Of Focus:
- CARPHA Member States
- CARICOM Countries
Year of Presentation:
2022
Objective: This study was a comprehensive comparative
analysis of the performance of EMQ and traditional MCQ
formats in the written medicine and therapeutics component
of final MBBS examination.
Design and Methods: An item analysis of 80 EMQs and 200 MCQs administered to 532 examinees across four campuses/site in different Caribbean countries during final MBBS medicine and therapeutics examination of 2019. Exam performance measures included central tendency, item discrimination, reliability, item difficulty, and distractor efficacy.
Results: For the 532 students who sat the exam, the highest, lowest, and mean (+SD) scores for EMQs were 93, 41, and 69.0 (+9.8); for MCQs, the respective values were 82, 41 and 62.7 (+7.4). The predictive value of EMQ and MCQ scores for overall failure was 0.67 (95% CI = 0.39, 0.87) and 0.89 (95% CI = 0.65, 0.98) respectively. There were no statistically significant differences in discrimination index (DI) scores by question type for any of the four cohorts. KR-20 coefficients for EMQs and MCQs ranged from 0.52 to 0.70 and 0.71 to 0.79, respectively. The proportion of questions with two or more functional distractors was consistently higher for MCQs than for EMQs in all four cohorts of students.
Conclusion: The wider spread of EMQ compared to MCQ scores suggests that the former are suitable for formative assessment. However, MCQ scores were more predictive of overall exam failure, which suggests that MCQs are more suitable for high-stakes assessments such as the final MBBS examination.