How to cite this article: Rojas-Armadillo ML, Jiménez-Báez MV, Chávez-Hernández MM, González-Fondón A. [Validation of an adverse event reporting system in primary care]. Rev Med Inst Mex Seguro Soc. . 2016 May-Jun;54(3):327-33.
Received: December 1st /2015
Judged: October 13rd 2015
María de Lourdes Rojas-Armadillo,a María Valeria Jiménez-Báez,b María Margarita Chávez-Hernández,c Araceli González-Fondónd
aCoordinación Delegacional de Planeación y Enlace Institucional
bCoordinación Auxiliar de Investigación en Salud
cCoordinación Auxiliar de Educación en Salud
dCoordinación Clínica de Medicina Familiar, Hospital General de Zona/Unidad de Medicina Familiar 16, Chetumal
a,b,cJefatura de Prestaciones Médicas, Cancún
Instituto Mexicano del Seguro Social, Quintana Roo, México
Todas las autoras pertenecen al Grupo de Investigación Clínico Epidemiológico del IMSS, Quintana Roo
Communication with: María Valeria Jiménez-Báez
Telephone: (998) 267 870
Background: Patient safety is a priority issue in health systems, due to the damage costs, institutional weakening, lack of credibility, and frustration on those who committed an error that resulted in an adverse event. There is no standardized instrument for recording, reporting, and analyzing sentinel or adverse events (AE) in primary care. Our aim was to design and validate a surveillance system for recording sentinel events, adverse events and near miss incidents in primary care.
Methods: We made a review of systems for recording and reporting adverse events in primary care. Then, we proposed an instrument to record these events, and register faults in the structure and process, in primary health care units in the Instituto Mexicano del Seguro Social.
Results. We showed VENCER-MF format to 35 subjects. Out of them, 100 % identified a failure in care process, 90 % recorded a sentinel event, 85 % identified the cause of this event, 75 % of them suggested some measures for avoiding the recurrence of adverse events. We used a Cronbach’s alpha of 0.6, p = 0.03.
Conclusion: The instrument VENCER-MF has a good consistency for the identification of adverse events.
Keywords: Sentinel health event; Primary health care
The Instituto Mexicano del Seguro Social (IMSS) is currently governed by a system of quality management and patient safety, using approved standards for hospital certification from the Consejo de Salubridad General, which were established by the Sistema Nacional de Certificación de Establecimientos de Atención Médica, formally established in 2007.1
The three-pronged approach to evaluating health care (structure-process-outcome) was established by Donabedian. The central premise of this approach is that quality care maximizes patient wellbeing after taking into account the balance of expected losses and gains.2,3
Efficiency and quality are two inseparable components of health care, so all waste during medical care means low quality.2,4
In 1913 the evaluation of health institutions first emerged in the United States; in 1926 in Australia and in 1953 in Canada. Latin America and Asia have models more aimed at the regulatory process than at evaluation.5 The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) proposes the concept and dimensions of quality from two points of view: doing things well and doing them correctly.5
Safety is defined as the set of organizational structures or processes that reduces the likelihood of adverse events resulting from exposure of the health care system to diseases and procedures (Agency for Health Care Quality Research). This the reduction to an acceptable minimum risk of unnecessary harm associated with health care.6 It is a key dimension in healthcare quality aimed at promoting safe practices to prevent the reactive culture that blames professionals for their errors.7
Safety culture is defined as the integrated pattern of individual and organizational behavior, based on shared beliefs and values, which seeks to minimize the damage that the patient could suffer as a result of the processes of caregiving.8
An adverse event is an unexpected event, unrelated to the natural history of disease, that was a consequence of the process of care. The damage resulting from measures taken during the delivery of health care that is not due to an underlying disease or injury is known as health care-associated damage.6 When these events are preventable, they receive the category of error and can be classified by the order they appear in the clinical act, their nature, or origin.9
A sentinel event is an unforeseen occurrence whose result is not related to the natural course of a disease, but to medical care that causes the patient's death or permanent loss of a function or organ; it can also be a surgery in the wrong place, with the wrong procedure, or the wrong patient. This concept also can include events that are considered as such by their severity and the characteristics of the unit, for example, the theft of a child. A near-miss is a medical error that did not produce an adverse event because it was detected on time.4
In the United States the Institute of Medicine (IOM) began the project "Quality of Health Care in America" in 1998. The report "To Err is Human: Building a Safer Health System" document the presence of 98,000 errors per year, higher than car accidents, breast cancer, or AIDS, so they conclude that a solid notification system is a key strategy to learn from mistakes and avoid recurrence.10
In Latin America, the IBEAS study developed in five countries, including Mexico, said that the incidence of adverse events was 11.85% and the preventability was 65%. Adverse events were related to care in 13.27%, medication use in 8.23%, nosocomial infections in 37.14%, some procedure in 28.69%, and diagnosis in 6.15%. All studies found that almost half of the adverse events may be preventable.11
International reports indicate that only 5% of adverse events are reported.12 There is still controversy about the characteristics of an ideal communication system, given that to succeed it would require a safety culture implanted in the organization.13
Among the ideal characteristics of an adverse event recording and reporting system, it must be confidential, non-punitive, independent of any authority with power to punish the reporter, must include expert analysis done on time, must have a system guide that reflects the improvement of the system, and must have responsiveness to disseminate recommendations.14 In all cases the objectives of the reporting system depend on the security requirements and needs of the program to be created, for it to be a useful tool in which the information obtained allows progress in care for patient safety and a chance to improve the system.
IMSS has in effect the Sistema de Vigilancia de Eventos Centinela, Eventos Adversos y Cuasifallas (VENCER II, Sentinel Event, Adverse Event, and Near-Miss Surveillance System), through the mechanism of identification, reporting, and root cause analysis for second and third level medical care units, where adverse events are recorded systematically at a national level.4 However, there are reports that indicate that 84% of staff do not know the program.15
There is so far no adverse event notification registration system for primary care in Mexico, much less a validated instrument that allows the systematic recording of such events. Safety research has focused on hospitals, although in recent years the interest in primary care has increased. This is very important, especially in emerging economies where primary care looms large in health care.12
Prevention practices in primary care are based on the areas with the highest record of adverse events, such as medication safety.16 In 2003 Rubin made a proposal to describe the classification of errors through validation with experts, and out of 940 registered errors it was found that most were secondary to prescription drugs, but only 6% were due to the adverse effects of medicines; 30% were errors in communication.17
In 2009 Escobar-Pérez et al., in a study of primary care, demonstrated that all adverse effects produce an increase in the average stay of patients in most services where they occur, therefore increasing the direct cost.3
The World Health Organization (WHO) conducted a literature review in 2008 of the methods and measurements used in primary care to investigate patient safety.18
Ruelas et al. identified that 29.9% of adverse events in 127 cases studied caused harm to patients. Of these, 21% occurred in primary care units, especially in the emergency department. 2.6% of these events originated in Quintana Roo.19
In 2008 at the Comisión Nacional de Arbitraje Médico (CONAMED) published the results of medical error reporting on its website. From the reports from 2005 to 2007, it was found that out of 200 uses only 166 were valid; only 9.1% was considered probable medical error, and the remainder were patient complaints. Of the errors, 35.7% corresponded to medications, 14.3% to patient falls, and 50% was associated in some way with surgical procedures.20
This is why the purpose of our study was to design and validate a surveillance system for recording sentry, adverse, and near-miss events in primary care units, given the need for reliable instruments for reporting adverse events.
This study was approved by the Local Committee on Health Research with COFEPRIS registration number 23012013-2301-44, in compliance with the current law on health research.
This is a descriptive study. After literature review on adverse incident recording and reporting systems for primary care in other countries in indexed journals, with minimum level II evidence and impact factor, a proposal was designed integrating the error record into the structure and errors in the process, including the health care process, professional competencies, and factors inherent to the patient and the culture of the organization.
A questionnaire was designed (content validity) for the Sistema de Registro y Notificación de Incidentes y Eventos Adversos para Unidades de Medicina Familiar (Adverse Event and Incident Reporting and Registration System for Family Medicine Units, VENCER-MF).
The questionnaire was subjected to construct validation based on expert review and five were included, of which one is a quality evaluator, three doctors are family medicine specialists, and one nurse is a primary care specialist. Based on the review of the questionnaire, brainstorming and the Delphi method were used to correct the VENCER-MF.
Items were accepted until obtaining a consensus of five out of five experts.
In order to proceed to determine the consistency of the VENCER-MF instrument and eliminate the factor of chance, researchers proceeded, after finishing its construction, to implement a pilot test, which consisted of applying the questionnaire to an unrelated group of 35 first semester-students of the Bachelor’s of Nursing who were taking the introduction course, who were asked for their permission to participate in the study by signing informed consent.
The classification of adverse events in the instrument was reported as sentinel event, adverse event, or near-miss.
The following variables were considered: sex, age, identified sentinel event, identified origin, identified the result of the event, identified other factors, identified the human factor, identified organizational failure, and implements improvements.
As for statistical analysis, for descriptive statistics, the frequencies of all the variables included were described; nominals and ordinals were described with percentages or proportions according to the case, and numerals were described by measures of central tendency and dispersion measures. As for inferential statistics, Cronbach’s alpha was used (Figures 1 and 2).
Figure 1 General outline of the study. Validation of an adverse event registration system in primary care
Instituto Mexicano del Seguro Social
VENCER II SYSTEM FOR UMF
Sentinel event, adverse event, and near-miss reporting
Based on the political Constitution of the Mexican United States, article 4o, Ley General de Salud, Art. 2 and 51, regulation of the General health law, in the field of
GENERAL INFORMATION ABOUT THE CASE
|PATIENT'S FULL NAME||SEX|
|SOCIAL SECURITY NO.||MAIN DX|
|I. SENTINEL EVENT.Unplanned event, result of medical care resulting in the death of the patient, permanent loss of function or organ, or wrong surgery, and other
the event is considered as SENTINEL by the policies of the establishment.
|How did the Sentinel Event originate?||In the diagnosis||What was the result?|
|In the correct identification of patient||In the treatment||Death|
|Prescription of drugs||Falls||Permanent loss of function|
|Application of drugs||Other||Loss of organ|
|Application of dose or type of vaccine|
|II. ADVERSE EVENT.Damage a result of medical care and not the baseline conditions of the patient.|
How did the adverse event originate?
|In the correct identification of patient||In the diagnosis||Falls|
|Prescription of drugs||In the treatment||Other|
|Application of drugs|
|Application of dose or type of vaccine|
|What happened in the adverse event?||How do you rate the severity of the consequences?|
|Did it produce consequences?||Quick solution|
|No||Requires additional treatment|
|If||Requires high complexity Tx|
|III. NEAR-MISS.Medical error that did not result in an adverse event because it was detected in time.|
If there was a near-miss, what type was it?
|In the correct identification of patient||In the diagnosis||Falls|
|Prescription of drugs||In the treatment||Other|
|Application of drugs||Mention which|
|Application of dose or type of vaccine|
|BRIEFLY DESCRIBE WHAT HAPPENED|
|INDICATE OTHER FACTORS THAT INFLUENCED DIRECTLY|
|Human factor||Doctor||Lack of resources|
|Intern||Failure or malfunction of equipment|
|Mention which||Mention which|
|MEASURES BE IMPLEMENTED TO CORRECT AND PREVENT THE EVENT?|
Figure 2 VENCER-MF instrument format of Instituto Mexicano del Seguro Social
The pilot test was conducted in 35 subjects, who were presented with a case and asked to report it in the VENCER-MF form. The mean age was 20.1 ± 2.8. 70% were women and 100% correctly identified the presence of an error in the health care process. 90% reported a sentinel event. 85% correctly identified the origin of the sentinel event, 90% reported the result of the sentinel event, 95% described what happened, 95% identified other factors that directly affect the adverse event, 90% identified failures in professional competency, 60% identified errors in the organization, 100% discerned factors inherent to the patient, and 75% suggested measures to prevent the recurrence of adverse events.
The internal consistency of the instrument was verified with Cronbach's alpha at 0.6, with p = 0.03 (Table I).
|Table I Internal consistency test (n = 35)|
|Identifies sentinel event||7.05||0.99||0.49||0.20|
|Identifies results of event||7.05||0.99||0.49||0.20|
|Identifies other factors||7.10||1.25||0.18||0.36|
|Identifies human factor||7.05||1.31||-0.01||0.44|
|Identifies organizational error||6.75||1.25||-0.09||0.56|
|µ = mean of scale if item is deleted; S2 = variance of scale if element is deleted; ® = corrected element-total correlation; α = Cronbach's alpha if item is deleted. p-value < 0.03 implied statistical significance|
The main purpose of the patient safety reporting system is to learn from the experience and use the results of the analysis and research data to formulate and disseminate recommendations for changes in the health system.21
To reduce the frequency of safety issues, it is necessary to understand their causes and devise methods to prevent or detect them before they cause harm to patients. Hence, learning from mistakes has become one of the objectives of the strategies for patient safety. One way to do this is to have a system for reporting safety problems or potential problems. Knowing in detail the chain of events culminating in a patient safety incident represents an opportunity to prevent possible occurrences of similar situations in other patients, and therefore should help reduce adverse events.22
Reporting systems are not intended to estimate the frequency of adverse events. They must be non-punitive systems and their goal should be continuous improvement. It has been found that these systems often get greater involvement from professionals when they are voluntary and anonymous,23 but more information and evidence is needed to confirm whether they do contribute to patient safety.24
Because it is considered necessary to facilitate the exchange and dissemination of regional experience, the regional office of the Pan American Health Organization (PAHO / Washington), in collaboration with PAHO / Mexico and the aforementioned CONAMED, did a study analyzing notification systems operating in Latin America. The different reporting and monitoring systems in some of the countries in the region were presented. Each of these systems responds to national goals and needs. The experiences of countries in the region and outside it show a variety of operational modalities of these systems, reflecting the diversity of existing alternatives.25
There is still no conclusive scientific evidence on the benefits of one system versus another, although there is a broad consensus on the learning and non-punitive nature that should characterize these systems.25
Systems that have been implemented in the centers of many countries do not replace other traditionally used strategies for analyzing the frequency of adverse events, epidemiological studies or information systems for nosocomial infections or adverse drug reactions;26 it is recommended that professionals receive the information quickly and clearly, both on the conditions of participation in the reporting system and the results of the analysis of reported cases.27
Although the culture of patient safety is widespread in primary care, studies have not yet yielded an effective strategy for this level of care.
While Australia, the US, and the UK have begun to build research programs on patient safety in primary care, reporting of adverse events in this type of care in our country is not sufficient. And it is at this level where the most care in the health systems happens. We can find in the literature several reasons that explain the low registered number, for example, non-recognition of adverse events, confusion over the different operational definitions, fear of possible punitive actions, concerns about anonymity and confidentiality of the information in the report, not having enough time during the workday, and distrust of the possible effect that the report might have.
The VENCER system in Mexico was developed starting in 2005 by the social security institution of greatest coverage in the country: Instituto Mexicano del Seguro Social (IMSS). The system proposes a proactive and preventive approach based on the analysis of sentinel events and the establishment of action plans to prevent recurrence, but only at the hospital level.4
The VENCER-MF instrument in our study has sufficient consistency for the identification and registration of adverse events. Applying the pilot to an unrelated population allowed us an approach to a validated recording system for reporting adverse events, of which there is no scientific precedent, probably due to the absence of a definitive and international consensus on terms and definitions for reporting adverse events.
This instrument took into consideration enough dimensions to cover the horizon of presentation of an adverse event, from its occurrence to decisions to improve health systems, which is a strength of our instrument that allows a full survey of what happens. The revised proposals so far only offer very open options that weaken the consistency, standardization, and presentation of the analysis, which directly affects the effectiveness of system improvement.
Among the weaknesses in our study, we noted the need to expand the population for the pilot to strengthen the consistency of our instrument.
This is the beginning of the patient safety research offered by the state of Quintana Roo, which features medical units similar to those of other states in the country.
Conflict of interest statement: The authors have completed and submitted the form translated into Spanish for the declaration of potential conflicts of interest of the International Committee of Medical Journal Editors, and none were reported in relation to this article.