Open Journal Systems

ORIGINAL CONTRIBUTIONS

Received: December 1^st /2015

Judged: October 13^rd 2015

Validation of an adverse event reporting system in primary care

María de Lourdes Rojas-Armadillo,^a María Valeria Jiménez-Báez,^b María Margarita Chávez-Hernández,^c Araceli González-Fondón^d

^aCoordinación Delegacional de Planeación y Enlace Institucional

^bCoordinación Auxiliar de Investigación en Salud

^cCoordinación Auxiliar de Educación en Salud

^dCoordinación Clínica de Medicina Familiar, Hospital General de Zona/Unidad de Medicina Familiar 16, Chetumal

^a,b,cJefatura de Prestaciones Médicas, Cancún

Instituto Mexicano del Seguro Social, Quintana Roo, México

Todas las autoras pertenecen al Grupo de Investigación Clínico Epidemiológico del IMSS, Quintana Roo

Communication with: María Valeria Jiménez-Báez

Telephone: (998) 267 870

Email: valeria.jimenezb@gmail.com

Background: Patient safety is a priority issue in health systems, due to the damage costs, institutional weakening, lack of credibility, and frustration on those who committed an error that resulted in an adverse event. There is no standardized instrument for recording, reporting, and analyzing sentinel or adverse events (AE) in primary care. Our aim was to design and validate a surveillance system for recording sentinel events, adverse events and near miss incidents in primary care.

Methods: We made a review of systems for recording and reporting adverse events in primary care. Then, we proposed an instrument to record these events, and register faults in the structure and process, in primary health care units in the Instituto Mexicano del Seguro Social.

Results. We showed VENCER-MF format to 35 subjects. Out of them, 100 % identified a failure in care process, 90 % recorded a sentinel event, 85 % identified the cause of this event, 75 % of them suggested some measures for avoiding the recurrence of adverse events. We used a Cronbach’s alpha of 0.6, p = 0.03.

Conclusion: The instrument VENCER-MF has a good consistency for the identification of adverse events.

Keywords: Sentinel health event; Primary health care

---

The Instituto Mexicano del Seguro Social (IMSS) is currently governed by a system of quality management and patient safety, using approved standards for hospital certification from the Consejo de Salubridad General, which were established by the Sistema Nacional de Certificación de Establecimientos de Atención Médica, formally established in 2007.¹

The three-pronged approach to evaluating health care (structure-process-outcome) was established by Donabedian. The central premise of this approach is that quality care maximizes patient wellbeing after taking into account the balance of expected losses and gains.^2,3

Efficiency and quality are two inseparable components of health care, so all waste during medical care means low quality.^2,4

In 1913 the evaluation of health institutions first emerged in the United States; in 1926 in Australia and in 1953 in Canada. Latin America and Asia have models more aimed at the regulatory process than at evaluation.5 The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) proposes the concept and dimensions of quality from two points of view: doing things well and doing them correctly.⁵

Safety is defined as the set of organizational structures or processes that reduces the likelihood of adverse events resulting from exposure of the health care system to diseases and procedures (Agency for Health Care Quality Research). This the reduction to an acceptable minimum risk of unnecessary harm associated with health care.⁶ It is a key dimension in healthcare quality aimed at promoting safe practices to prevent the reactive culture that blames professionals for their errors.⁷

Safety culture is defined as the integrated pattern of individual and organizational behavior, based on shared beliefs and values, which seeks to minimize the damage that the patient could suffer as a result of the processes of caregiving.⁸

An adverse event is an unexpected event, unrelated to the natural history of disease, that was a consequence of the process of care. The damage resulting from measures taken during the delivery of health care that is not due to an underlying disease or injury is known as health care-associated damage.6 When these events are preventable, they receive the category of error and can be classified by the order they appear in the clinical act, their nature, or origin.⁹

A sentinel event is an unforeseen occurrence whose result is not related to the natural course of a disease, but to medical care that causes the patient's death or permanent loss of a function or organ; it can also be a surgery in the wrong place, with the wrong procedure, or the wrong patient. This concept also can include events that are considered as such by their severity and the characteristics of the unit, for example, the theft of a child. A near-miss is a medical error that did not produce an adverse event because it was detected on time.⁴

In the United States the Institute of Medicine (IOM) began the project "Quality of Health Care in America" ​​ in 1998. The report "To Err is Human: Building a Safer Health System" document the presence of 98,000 errors per year, higher than car accidents, breast cancer, or AIDS, so they conclude that a solid notification system is a key strategy to learn from mistakes and avoid recurrence.¹⁰

In Latin America, the IBEAS study developed in five countries, including Mexico, said that the incidence of adverse events was 11.85% and the preventability was 65%. Adverse events were related to care in 13.27%, medication use in 8.23%, nosocomial infections in 37.14%, some procedure in 28.69%, and diagnosis in 6.15%. All studies found that almost half of the adverse events may be preventable.¹¹

International reports indicate that only 5% of adverse events are reported.¹² There is still controversy about the characteristics of an ideal communication system, given that to succeed it would require a safety culture implanted in the organization.¹³

Among the ideal characteristics of an adverse event recording and reporting system, it must be confidential, non-punitive, independent of any authority with power to punish the reporter, must include expert analysis done on time, must have a system guide that reflects the improvement of the system, and must have responsiveness to disseminate recommendations.¹⁴ In all cases the objectives of the reporting system depend on the security requirements and needs of the program to be created, for it to be a useful tool in which the information obtained allows progress in care for patient safety and a chance to improve the system.

IMSS has in effect the Sistema de Vigilancia de Eventos Centinela, Eventos Adversos y Cuasifallas (VENCER II, Sentinel Event, Adverse Event, and Near-Miss Surveillance System), through the mechanism of identification, reporting, and root cause analysis for second and third level medical care units, where adverse events are recorded systematically at a national level.⁴ However, there are reports that indicate that 84% of staff do not know the program.¹⁵ 

There is so far no adverse event notification registration system for primary care in Mexico, much less a validated instrument that allows the systematic recording of such events. Safety research has focused on hospitals, although in recent years the interest in primary care has increased. This is very important, especially in emerging economies where primary care looms large in health care.¹²

Prevention practices in primary care are based on the areas with the highest record of adverse events, such as medication safety.¹⁶ In 2003 Rubin made a proposal to describe the classification of errors through validation with experts, and out of 940 registered errors it was found that most were secondary to prescription drugs, but only 6% were due to the adverse effects of medicines; 30% were errors in communication.¹⁷

In 2009 Escobar-Pérez et al., in a study of primary care, demonstrated that all adverse effects produce an increase in the average stay of patients in most services where they occur, therefore increasing the direct cost.³

The World Health Organization (WHO) conducted a literature review in 2008 of the methods and measurements used in primary care to investigate patient safety.¹⁸

Ruelas et al. identified that 29.9% of adverse events in 127 cases studied caused harm to patients. Of these, 21% occurred in primary care units, especially in the emergency department. 2.6% of these events originated in Quintana Roo.¹⁹

In 2008 at the Comisión Nacional de Arbitraje Médico (CONAMED) published the results of medical error reporting on its website. From the reports from 2005 to 2007, it was found that out of 200 uses only 166 were valid; only 9.1% was considered probable medical error, and the remainder were patient complaints. Of the errors, 35.7% corresponded to medications, 14.3% to patient falls, and 50% was associated in some way with surgical procedures.²⁰

This is why the purpose of our study was to design and validate a surveillance system for recording sentry, adverse, and near-miss events in primary care units, given the need for reliable instruments for reporting adverse events.

Methods

This study was approved by the Local Committee on Health Research with COFEPRIS registration number 23012013-2301-44, in compliance with the current law on health research.

This is a descriptive study. After literature review on adverse incident recording and reporting systems for primary care in other countries in indexed journals, with minimum level II evidence and impact factor, a proposal was designed integrating the error record into the structure and errors in the process, including the health care process, professional competencies, and factors inherent to the patient and the culture of the organization.

A questionnaire was designed (content validity) for the Sistema de Registro y Notificación de Incidentes y Eventos Adversos para Unidades de Medicina Familiar (Adverse Event and Incident Reporting and Registration System for Family Medicine Units, VENCER-MF). 

The questionnaire was subjected to construct validation based on expert review and five were included, of which one is a quality evaluator, three doctors are family medicine specialists, and one nurse is a primary care specialist. Based on the review of the questionnaire, brainstorming and the Delphi method were used to correct the VENCER-MF.

Items were accepted until obtaining a consensus of five out of five experts.

In order to proceed to determine the consistency of the VENCER-MF instrument and eliminate the factor of chance, researchers proceeded, after finishing its construction, to implement a pilot test, which consisted of applying the questionnaire to an unrelated group of 35 first semester-students of the Bachelor’s of Nursing who were taking the introduction course, who were asked for their permission to participate in the study by signing informed consent.

The classification of adverse events in the instrument was reported as sentinel event, adverse event, or near-miss.

The following variables were considered: sex, age, identified sentinel event, identified origin, identified the result of the event, identified other factors, identified the human factor, identified organizational failure, and implements improvements.

As for statistical analysis, for descriptive statistics, the frequencies of all the variables included were described; nominals and ordinals were described with percentages or proportions according to the case, and numerals were described by measures of central tendency and dispersion measures. As for inferential statistics, Cronbach’s alpha was used (Figures 1 and 2).

Results

The pilot test was conducted in 35 subjects, who were presented with a case and asked to report it in the VENCER-MF form. The mean age was 20.1 ± 2.8. 70% were women and 100% correctly identified the presence of an error in the health care process. 90% reported a sentinel event. 85% correctly identified the origin of the sentinel event, 90% reported the result of the sentinel event, 95% described what happened, 95% identified other factors that directly affect the adverse event, 90% identified failures in professional competency, 60% identified errors in the organization, 100% discerned factors inherent to the patient, and 75% suggested measures to prevent the recurrence of adverse events.

The internal consistency of the instrument was verified with Cronbach's alpha at 0.6, with p = 0.03 (Table I).

Discussion

The main purpose of the patient safety reporting system is to learn from the experience and use the results of the analysis and research data to formulate and disseminate recommendations for changes in the health system.²¹

To reduce the frequency of safety issues, it is necessary to understand their causes and devise methods to prevent or detect them before they cause harm to patients. Hence, learning from mistakes has become one of the objectives of the strategies for patient safety. One way to do this is to have a system for reporting safety problems or potential problems. Knowing in detail the chain of events culminating in a patient safety incident represents an opportunity to prevent possible occurrences of similar situations in other patients, and therefore should help reduce adverse events.²²

Reporting systems are not intended to estimate the frequency of adverse events. They must be non-punitive systems and their goal should be continuous improvement. It has been found that these systems often get greater involvement from professionals when they are voluntary and anonymous,²³ but more information and evidence is needed to confirm whether they do contribute to patient safety.²⁴

Because it is considered necessary to facilitate the exchange and dissemination of regional experience, the regional office of the Pan American Health Organization (PAHO / Washington), in collaboration with PAHO / Mexico and the aforementioned CONAMED, did a study analyzing notification systems operating in Latin America. The different reporting and monitoring systems in some of the countries in the region were presented. Each of these systems responds to national goals and needs. The experiences of countries in the region and outside it show a variety of operational modalities of these systems, reflecting the diversity of existing alternatives.²⁵

There is still no conclusive scientific evidence on the benefits of one system versus another, although there is a broad consensus on the learning and non-punitive nature that should characterize these systems.²⁵

Systems that have been implemented in the centers of many countries do not replace other traditionally used strategies for analyzing the frequency of adverse events, epidemiological studies or information systems for nosocomial infections or adverse drug reactions;²⁶ it is recommended that professionals receive the information quickly and clearly, both on the conditions of participation in the reporting system and the results of the analysis of reported cases.²⁷

Although the culture of patient safety is widespread in primary care, studies have not yet yielded an effective strategy for this level of care.

While Australia, the US, and the UK have begun to build research programs on patient safety in primary care, reporting of adverse events in this type of care in our country is not sufficient. And it is at this level where the most care in the health systems happens. We can find in the literature several reasons that explain the low registered number, for example, non-recognition of adverse events, confusion over the different operational definitions, fear of possible punitive actions, concerns about anonymity and confidentiality of the information in the report, not having enough time during the workday, and distrust of the possible effect that the report might have.  

The VENCER system in Mexico was developed starting in 2005 by the social security institution of greatest coverage in the country: Instituto Mexicano del Seguro Social (IMSS). The system proposes a proactive and preventive approach based on the analysis of sentinel events and the establishment of action plans to prevent recurrence, but only at the hospital level.⁴

The VENCER-MF instrument in our study has sufficient consistency for the identification and registration of adverse events. Applying the pilot to an unrelated population allowed us an approach to a validated recording system for reporting adverse events, of which there is no scientific precedent, probably due to the absence of a definitive and international consensus on terms and definitions for reporting adverse events. 

This instrument took into consideration enough dimensions to cover the horizon of presentation of an adverse event, from its occurrence to decisions to improve health systems, which is a strength of our instrument that allows a full survey of what happens. The revised proposals so far only offer very open options that weaken the consistency, standardization, and presentation of the analysis, which directly affects the effectiveness of system improvement.

Among the weaknesses in our study, we noted the need to expand the population for the pilot to strengthen the consistency of our instrument.

This is the beginning of the patient safety research offered by the state of Quintana Roo, which features medical units similar to those of other states in the country.

References

1. Consejo de Salubridad General. Estándares para la certificación de clínicas de atención primaria y consulta de especialidades 2012. México, DF: Comisión para la Certificación de Establecimientos de Atención Médica; 2012. p. 85.
2. Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q. 2005;83(4):691-729.
3. Escobar-Pérez B, Lobo-López MM, Sala-Turrens J. Calidad Asistencial, seguridad del paciente e ineficiencia: estudio de campo en un hospital de primer nivel. Revista del Instituto Internacional de Costos. 2009;5:271-305.
4. Instituto Mexicano del Seguro Social. Manual del sistema Vencer II. Sistema de vigilancia de eventos centinela, eventos adversos y cuasifallas. México D.F.: Instituto Mexicano del Seguro Social; 2011.
5. Ministerio de Sanidad y Consumo. Prácticas Seguras Simples. Recomendadas por agentes gubernamentales para la prevención de Efectos Adversos (EA) en los pacientes atendidos en hospitales. Madrid: Ministerio de Sanidad y Consumo; 2008.
6. Organización Mundial de la Salud [OMS]. Marco conceptual de la clasificación internacional para la seguridad del paciente. Informe Técnico Definitivo. Patient Safety A word alliance for safer Health Care. OMS; 2009; pp. 1-160.
7. Aranaz JM, Moya C. [Patient safety and health care quality]. Rev Calid Asist. 2011;26(6):331-2.
8. Zwart DLM, Langelaan M, van de Vooren RC, Kuyvenhoven MM, Kalkman CJ, Verheij TJM, et al. Patient safety culture measurement in general practice. Clinimetric properties of ‘SCOPE’. BMC Fam Pract. 2011;12:117.
9. Francesc B. Seguridad clínica en atención primaria. Los errores médicos (II). Aten Primaria. 2011;711.
10. Donaldson MS. An Overview of To Err is Human: Re-emphasizing the Message of Patient Safety. In: Hughes RG, ed. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville (MD): Agency for Healthcare Research and Quality (US). 2008 Apr.
11. Ministerio de Sanidad, Política Social e Igualdad/Organización Mundial de la Salud. Estudio IBEAS. Prevalencia de efectos adversos en hospitales de Latinoamérica. Madrid: Ministerio de Sanidad y Política Social; 2009.
12. Gaal S, Verstappen W, Wensing M. What do primary care physicians and researchers consider the most important patient safety improvement strategies? BMC Health Serv Res. 2011;11:102.
13. Bartolomé A, Gómez-Arnau JI, García-del Valle S, González-Arévalo G, Santa-Úrsula JA, Hidalgo I. Seguridad del paciente y sistemas de comunicación de incidentes. Rev Calidad Asistencial. 2005;20(4):228-34.
14. Bañeres J, Cavero E, López L, Orrego C, Suñol R. Sistemas de registro y notificación de incidentes y eventos adversos. Agencia de calidad del Sistema Nacional de Salud [Internet]. Madrid: Ministerio de Sanidad y Consumo; 2007. Available from: http://www.msssi.gob.es/organizacion/sns/planCalidadSNS/pdf/excelencia/opsc_sp3.pdf
15. Juárez-Pérez H, Durán-Muñoz C. Eventos centinela y la notificación por el personal de enfermería. Rev Enferm Inst Mex Seguro Soc. 2009;17(1):39-44.
16. Hall KB, Tett SE, Nissen LM. Perceptions of the influence of prescription medicine samples on prescribing by family physicians. Med Care. 2006 Apr; 44(4):383-7.
17. Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice: development of an error classification and pilot study of a method for detecting errors. Qual Saf Health Care. 2003;12:443-7.
18. Dovey S, Makeham M, Runciman W, Larizgoitia I. Methods and measures used in primary care patient safety research. Results of a literature Review. Geneva: World Health Organization; 2008; pp. 1-49.
19. Ruelas-Barajas E, Tena-Tamayo C, Sánchez-González J, Sarabia-González O, Hernández-Gamboa LE, Campos-Castolo E. Eventos adversos identificables en las quejas médicas. Cir Ciruj. 2008;76:153-60.
20. Campos Castolo EMCJ, Arturo. Medical error report as an strategy for the prevention of adverse events. Revista Conamed. 2008;13:17.
21. World Health Organization. Draft guidelines for adverse events reporting and learning system. World alliance for patient safety. 2005. Available from: www.who.int/patientsafety
22. Cooper JB, Newbower RS, Long CD, McPeek B. Preventable anesthesia mishaps: a study of human factors. Anesthesiology. 1978;49:399-406.
23. Vincent C. Incident reporting and patient safety. BMJ. 2007;334:51.
24. Snijders C, van Lingen RA, Molendijk A, Fetter WPF. Incident and errors in neonatal intensive care: a review of the literature. Arch Dis Child Fetal Neonatal, 2007 Sep; 92(5): F391–F398. doi: 10.1136/adc.2006.106419
25. Organización Panamericana de la Salud. Sistemas de notificación de incidentes en América Latina. Washington, DC: OPS, 2013.
26. Aranaz J. Acerca de los sistemas de notificación y registro de sucesos adversos. Rev Calidad Asistencial. 2009; 24:1-2.
27. Mira JJ, Cho M, Montserrat D, Rodríguez J, Santacruz J. Elementos clave en la implantación de sistemas de notificación de eventos adversos hospitalarios en América Latina. Rev Panam Salud Pública. 2013;33(1):1-7.

Enlaces refback

• No hay ningún enlace refback.