Reliability and validity of an instrument to evaluate integral clinical competence in medical residents
Main Article Content
Keywords
Evaluation, Clinical competence, Medical residents
Abstract
Background: The evaluation of clinical competence in medical residents is a complex procedure. Teachers need reliable and valid instruments to evaluate objectively the, clinical competence. The aim of this study was to determine the reliability and validity of an instrument designed to evaluate the clinical competence of medical residents.
Methods: We designed an instrument taking into consideration every part of the clinical method, and three different levels of competence were determined for each one. The instrument was examined with regards to the clarity, pertinence and sufficiency of each clinical indicator by five expert pediatricians. The instrument was finally constituted by 11 indicators. Each resident was evaluated independently by three pediatricians.
Results: A total of 651 measurements were done in 234 residents. The instrument distinguished between extreme groups, had a value of Cronbach´s alpha of 0.778 and the factorial analysis set apart two factors: clinical competence and complementary competences. No statistical differences were found between evaluators neither in global evaluation or in each indicator.
Conclusion: The instrument here proposed is valid and reliable. It may be used in formative evaluation of medical residents in clinical specialization programs.
References
Fraser S., Greenhalgh T. Coping with complexity: educating for capability. BMJ 2001; 323: 799-803.
Regehr G. Kevin E., Ginsburg S. et al. Assessment in postgraduate medical education: trens and issues in assessment in the workplace. Members of the FMEC PG consortium; 2011.
Sloan DA, Donelly MB, Schwarts RW. Strodel WE. The Objetive structured clinical examination. The new gold standard for evaluating posgraduate clinical performance. Annals of Surgery 1995; 222(6):735-42
Hull AL, Hodder MD, Berger B, Ginsberg D, Lindheim N, Quan J.Validity of three clinical perfomance assessments of International Medicine Clerks. Acad Med 1995; 70(6): 517-522.
Harden RM, Gleeson FA. Assessment of Clinical Competence Using an Objetive Structured Clinical Examination (OSCE) Med Educ 1979;13: 41-54.
Dupras DM, Li JT. Use of an Objetive Structured Clinical Competence Acad Med 1995;70 (11): 1029-34.
Selby C, Osman L, Davis M, Lee M. How to do it. Set up and run.an objetive structured clinical exam. Br Med Journal. 1995; 310: 1187-90.
Kogan J., Holmboe E., Hauer K. Tools for direct observation and assessment of clinical skills of medical trainees. A systematic review. JAMA 2009; 302(12): 1316-1326.
Cook D., Bechman T., Mandrekar J, et al. Internal structure of mini-CEX scores for internal medicine residents: factor analysis and generalizability. Adv Health Sci Educ 2010; 15(5): 633-645
Ginsburg S., Mciloy J., Oulanova O., et al. Toward authentic clinical evaluation: pitfalls in the pursuit of competency. Academic Medicine 2010; 85(5): 780-786
Gingerich A., Regehr G., Eva K. Rater-based assessments as social judgements:rethinking the etiology of rater errors. Academic Medicine 2011; 86(10): 51-57
Kogan J., Conforti L., Bernabeo E, et al. Opening the black box of clinical skills assessment via observation: a conceptual model. Medical Education 2011; 45: 1048-1060.
Kogan J., Hess B., ConfortiL, et al. What drives faculty ratings of residents´clinical skills? The impact of faculty´s own clinical skills. Academic Medicine 2010; 85(10): 525-528.
Margolis M., Clauser B., Cuddy M, et al. Use of mini-clinical evaluation exercise to rate examinee performance on a multiple-station clinical skills examination: a validity study. Academic Medicine 2006; 81(10): 556-560
Epstein RM. Assessment in medical education. N Engl J Med 2007; 356 (4): 387-96.
Van der Vleuten C, Swanson DB. Assessment of clinical skills with standardized patients: state of the art. Teach Learn Med 1990; 2: 58–76.
Hays RB, Van der Vleuten C, Fabb WE, Spike NA. Longi- tudinal reliability of the Royal Australian College of General Practitioners certification examination. Med Educ 1995; 29: 317-321.
Page G, Bordage G, Allen T. Developing key-feature problems and examinations to assess clinical decision- making skills. Acad Med 1995; 70: 194-201.
Challis M. AMEE medical education guide no. 11: Portfolio- based learning and assessment in medical education. Med Teach 1999; 21: 370-386.
Van Der Vleuten C. Validity of final examinations in under- graduate medical training. BMJ 2000; 321: 1217-1219.
Accreditation Council for Graduate Medical Education (ACGME)-American Board of Medical Specialties (ABMS). ACGME/ABMS Joint Initiative Toolbox of Assessment Methods Version 1.1 September 2000. p. 1-3.
Trejo Mejia JA., Martinez Gonzalez A., Mendz Ramirez I, et al. Evaluación de la competencia clínica con el examen clínico objetivo estructurado en el internado médico de la Universidad Nacional Autónoma de México. Gac Med Mex 2014; 150: 8-17
Ponce de Leon-Castañeda ME., Ortiz-Montalvo A., Bonilla-Gonzalez I., et al. Evaluacion integral de la competencia clínica a través del examen profesional. Arch Med Fam 2006; 8(2):119-129
Soler-Huerta E., Sabido-Sighler C., Sainz-Vazquez L., et al. Confiabilidad de un instrumento para evaluar la aptitud clínica en residentes de medicina familiar. Arch Med Fam 2005; 7(1): 14-17
Pantoja Palmeros M., Barrera Monita J., Insfran Sanchez M. Instrumento para evaluar aptitud clínica en anestesiología. Rev Med Imss 2003; 41(1): 15-22.
Degollado Bardales L., Leyva Gonzalez FA., Viniegra Velazquez L., et al. Competencia clínica de los residentes de anestesiología en pacientes que requieren atención médica en un hospital de especialidades. Gac Med Mex 2009; 145(6): 481-489
Mendez Zarandona AP., Gutierrez Camacho C., Peña Alonso R. Estudio piloto para la validación de un instrumento de evaluación de habilidades y destrezas clínicas en pediatras en formación. Disponible en: http://www.facmed.unam.mx/sem/presentaciones/ARF/EV1.pdf