Piloting an outcome-based programme evaluation tool in undergraduate medical education.
AIMS
Different approaches to performance-oriented allocation of resources according to teaching quality are currently being discussed within German medical schools. The implementation of these programmes is impeded by a lack of valid criteria to measure teaching quality. An assessment of teaching quality should include structural and procedural aspects but focus on learning outcome itself. The aim of this study was to implement a novel, outcome-based evaluation tool within the clinical phase of a medical curriculum and address differences between the novel tool and traditional evaluation methods.


METHODS
Student self-assessments before and after completion of a teaching module were used to compute performance gains for specific learning objectives. Mean performance gains in each module were compared to student expectations before the module and data derived from a traditional evaluation tool using overall course ratings at the end of the module.


RESULTS
A ranking of the 21 modules according to computed performance gains yielded entirely different results than module rankings based on overall course ratings. There was no significant correlation between performance gain and overall ratings. However, the latter were significantly correlated to student expectations before entering the module as well as structural and procedural parameters (Pearson's r 0.7-0.9).


CONCLUSION
Performance gain computed from comparative self-assessments adds an important new dimension to course evaluation in medical education. In contrast to overall course ratings, the novel tool is less heavily confounded by construct-irrelevant factors. Thus, it appears to be more appropriate than overall course ratings in determining teaching quality and developing algorithms to guide performance-oriented resource allocation in medical education.
['Raupach|Tobias|T|', 'Schiekirka|Sarah|S|', 'Münscher|Christian|C|', 'Beißbarth|Tim|T|', 'Himmel|Wolfgang|W|', 'Burckhardt|Gerhard|G|', 'Pukrop|Tobias|T|']
D000124:Achievement D001291:Attitude of Health Personnel D003479:Curriculum D004504:Education, Medical, Undergraduate D005858:Germany D006801:Humans D017063:Outcome Assessment, Health Care D010865:Pilot Projects D015397:Program Evaluation D012647:Self-Assessment D013223:Statistics as Topic D013337:Students, Medical D013663:Teaching
evaluation self-assessment performance learning objective clinical studies congruence