P-37 An evaluation of the unified MBBS Exit Examination at the University of the West Indies
Author(s):
R Maharaj, S Motilal, D Ramnarine, M Fernandes, B Sa
Year of Presentation:
2026
Objective: In 2025, the University of the West Indies replaced discipline-specific final examinations with a unified exit examination combining a written paper and a multi-station OSCE. This study evaluated the psychometric
performance of the unified examination, focusing on reliability, OSCE circuit comparability, and the relationship
between written and clinical performance.
Methods: This was a retrospective review of de-identified assessment data from all candidates who sat the unified final MBBS examination at the St Augustine Campus in May/ June 2025. The examination comprised a 320-item single best answer multiple choice paper and a 17-station OSCE delivered concurrently across seven identical circuits. Intercircuit differences were examined using one-way ANOVA. Reliability was assessed using Cronbach’s alpha and Generalizability Theory, with decision-study modelling to estimate the number of stations required for high-stakes reliability. The relationship between written and OSCE scores was examined using Pearson’s correlation coefficient.
Results: De-identified written and OSCE scores from 157 candidates were analysed. Of the 320 multiple choice questions, 26 had a negative point biserial coefficient (PBSC) and a further 131 had a PBSC of 0.2 and hence were deemed acceptable regarding discrimination index. While 16 of 17 individual stations demonstrated statistically significant inter-circuit differences, variability was substantially attenuated when performance was aggregated across stations. Overall OSCE reliability was moderate (Cronbach’s alpha 0.72; G-coefficient 0.72; phi coefficient 0.69). Decision-study modelling indicated that increasing the number of stations would be required to achieve reliability appropriate for high-stakes assessment. Written and OSCE scores demonstrated a good positive correlation (r = 0.70, P<0.001).
Conclusion: A unified final MBBS examination is feasible and psychometrically defensible in large cohorts but requires adequate OSCE station sampling to support highstakes decision making.