Open Journal Systems

ORIGINAL CONTRIBUTIONS

How to cite this article: Panduro A, Ojeda-Granados C, Rivera H, Romana S. Índice h y productividad científica de investigadores en medicina de la Universidad de Guadalajara, México. Rev Med Inst Mex Seguro Soc. 2015;53(6):716-21.

PubMed: http://www.ncbi.nlm.nih.gov/pubmed/26506489

Received: April 4^th 2014

Accepted: October 28^th 2014

H Index and scientific output of researchers in medicine from the University of Guadalajara, Mexico

Arturo Panduro,^a,b Claudia Ojeda-Granados,^a Horacio Rivera,^c Sonia Romana,^d

^aServicio de Biología Molecular en Medicina, Antiguo Hospital Civil “Fray Antonio Alcalde”

^bDepartamento de Clínicas Médicas, Centro Universitario de Ciencias de la Salud, Universidad de Guadalajara

^cCentro de Investigación Biomédica de Occidente, División de Genética Humana, Instituto Mexicano del Seguro Social

^dDepartamento de Biología Molecular en Medicina y Genómica, Centro Universitario de Ciencias de la Salud, Universidad de Guadalajara

Guadalajara, Jalisco, México

Comunicación con: Arturo Panduro

Teléfono y fax: 52 (33) 3614 7743

Correo electrónico: apanduro@prodigy.net.mx

Objective: To evaluate by the h index the scientific output of researchers from the University of Guadalajara who belong to the Sistema Nacional de Investigadores in the field of Medicine and Health Sciences.

Methods: Researchers from the University of Guadalajara were selected from the Active SNI Researchers 2013 file. The scientific output of researchers in the fields of Medicine/Health Sciences and Biology/Chemistry was evaluated using the h index estimated by the Scopus website. A sample of capital researchers and Emeritus scientists was taken to perform the same procedure and compare data.

Results: The total number of researchers in the University of Guadalajara who are members of the SNI was 711, of which 67.2 % were level I and in less proportion were level II and III. Only 24.2 % of them were classified in the fields of Medicine/Health Sciences and Biology/Chemistry. The average h index value of researchers level I, II and III in Medicine/Health Sciences field was 5.4, 10.5 and 14.5, respectively. Capital and Emeritus scientists had an average h index of 23.4 and 19.8, respectively.

Conclusion: The h index measures the quantity and quality of the scientific output and it also avoids bias in the evaluation process. It should be useful for future evaluations of the SNI members and for medical doctors who sign up for the National Academy of Medicine

Keywords: Mexico, Citation databases, Bibliometrics, Biomedical research.

---

The most common parameters used to assess a researcher’s performance are the number of papers, the rate of citations per paper, and the impact factor (IF) of the scientific journals where these papers are published.

The IF of any given journal depends on the number of citations obtained by the papers published therein during the last two years. Thus, the higher the number of citations, the higher is the impact factor of the journal.¹ Nevertheless, one limitation of this evaluation parameter is that a scientific paper could be accepted to be published in a high-IF journal, but that does not guarantee a high number of citations. For example, one of the most widely-known journals worldwide, Nature, through a self-analysis of its 2005 impact factor, noticed that only 25% of their papers were widely cited to reach its high-IF.² Conversely, papers published in low-IF journals could have a higher number of citations than those published in high-IF journals.

In 1984, the Federal Government established, via the CONACYT (acronym in spanish for the National Council for Science and Technology), a national researcher organization known as the SNI (Sistema Nacional de Investigadores) in order to recognize and support the work of Mexican scientists.³ Within the SNI, there are different categories: candidate, levels ranging from I to III and the distinction of Emeritus scientist. The SNI also has seven different fields of knowledge: 1. Physics-Mathematics and Earth Sciences (PMES), 2. Biology and Chemistry (BC), 3. Medicine and Health Sciences (MHS), 4. Humanities and Behavioral Sciences (HBS), 5. Social Sciences (SS), 6. Biotechnology and Agricultural Sciences (BAS), 7. Engineering (ENG).

Each one has a peer-reviewer committee which assesses the scientific output of the researcher who requests accreditation. Both MHS and BC fields have similar evaluation process, in which the number of papers published in indexed journals and the number of citations are paramount.

Consequences of evaluations based largely on the number of publications in indexed journals are the excessive amount of co-authors and the high number of papers with a low number of citations and impact. Given these circumstances, in 2005 physicist Jorge E. Hirsch proposed the h index, or Hirsch index,⁴ which measures objectively the direct impact of a researcher’s work, both quantitatively and qualitatively.^4,5

A scientist’s h index is calculated based on his/her number of papers and the number of citations for each one.4 For example, a researcher could have 10, 20 or 50 papers, but if none of the papers is cited, the h index will be “0”. On the other hand, if a researcher has only one paper but with at least one citation, the h index will be 1; 2 papers with two citations each, h index = 2, and so on. This evaluation allows a more objective assessment of a researcher’s work since there is a balance between quantity and quality of his/her scientific output.

Furthermore, 20 years ago, a public policy mediated by the CONACYT was to decentralize the scientific research from Mexico, Distrito Federal (Capital city) to other states of the country. Given this situation, a new stage for the University of Guadalajara (UG) began with the founding of new thematic campuses, research teams and the development of novel graduate programs.⁶ During this period, the number of UG researchers in the SNI classified in the fields of MHS and BC rose from less than a dozen to over a hundred. Such an exponential growth prompted this study aimed to evaluate by the h index the scientific output of UG researchers belonging to the SNI and grouped in the MHS and BC fields. For comparison, a small sample of SNI level III researchers located in Mexico, D.F. and Emeritus scientists was also studied.

Methods

A descriptive cross-sectional study was conducted. The electronic file Active Researchers 2013 was downloaded in December 2013 from the SNI web page (http://www.conacyt.mx/index.php/el-conacyt/sistema-nacional-de-investigadores). Researchers from the UG were selected and first described by total number of candidates and SNI levels I, II and III, researchers per SNI’s knowledge fields and per UG campuses

Since the aim of this study was to evaluate the h index mainly in the field of Medicine and Health Sciences, we focused on analyzing the researchers from two campuses of the UG, as well as those registered in two SNI’s knowledge fields. In the group of researchers from the UG (candidates were not considered), two selections were conducted. In the first stage of analysis, researchers who work at the Health Sciences Campus (CUCS) and the Biological and Agricultural Sciences Campus (CUCBA) were selected and for the second stage, only those researchers classified in MHS or BC SNI’s field were analyzed.

A small sample of Emeritus scientists as well as SNI level III researchers located in Mexico, D.F., “capital researchers”, was randomly taken also from the file mentioned above in order to later compare data obtained. In all cases, the Scopus database from Elsevier was used to examine the total number of papers, citations and h index for each researcher.

The Scopus database nearly retrieves the total amount of research papers of the scientists (since it excludes non-index journals), regardless of the initial period of his/her scientific productivity. Nevertheless, it includes and calculates the citations and h index, respectively, since 1996 onwards.

Given that the majority of researchers with SNI level III (UG and capital researchers) and Emeritus scientists are of older age than those with level I or II, their h index was re-calculated. For this estimation, the total number of papers with its citations reported by Scopus starting at the beginning of each researcher’s productivity was examined.

For the statistical analysis qualitative variables were analyzed with frequencies and percentages and quantitative variables with average and ranges.

Results

The total number of researchers from the UG registered in the SNI 2013 database was 711. As shown in Figure 1, SNI members with level I (67.2%) prevail over the levels II (12.7%) and III (3.8%).

Figure 2 shows the percentage of researchers registered in each of the SNI’s knowledge fields, where researchers evaluated by the Humanities and Behavioral Sciences (24%) as well as the Social Sciences commissions (23.1%) are predominant. They are followed by researchers in the fields of Medicine and Health Sciences (14.5%) and Engineering (11.8%). These researchers are distributed throughout the fifteen thematic campuses of the UG and other Research Centers as shown in Figure 3.

When researchers from CUCS and CUCBA were evaluated in the first stage of the analysis, it was noteworthy that they were registered not only in MHS and BC fields, but also in other SNI’s knowledge fields. Thus, table I depicts the number of researchers in each one of these knowledge fields, their SNI level as well as the average of papers published in the indexed journals, citations and h index.

Table II describes UG researchers per SNI level along with their average number and range of papers, citations and h index, of those registered in the SNI’s knowledge fields of BC (n = 53) and MHS (n = 88).

While the average number of papers, citations and h index of researchers with SNI level I in both areas is similar, the average value of these same parameters is higher in researchers with SNI level II in the area of MHS (55 vs. 26.3, 473 vs. 245 and 10.5 vs. 8.3 respectively). Researchers with SNI level III in the field of MHS had in average, 94 papers, 759.6 citations and h index of 14.5; in the BC field, there were no UG researchers with SNI level III.

Globally, the scientific output in terms of average number of papers and citations was lower in the UG researchers with SNI level III registered in the field of MHS, compared to the SNI level III capital researchers and Emeritus scientists. However, the h index range of Emeritus scientists was similar to that of level III UG researchers (16-25 vs. 10-21) but lower than that of capital researchers (16-25 vs. 11-38) as shown in table III. However, these parameters were not evaluated by age or certain time.

Discussion and conclusions

In the last fifteen years, the growth of research in the UG has been focused on the development of graduate programs, research teams and the incorporation of graduate alumni as young researchers. Furthermore, many professors who were already employees at the UG had the opportunity to pursue a graduate degree and ascend to a higher level of research productivity. In conjunction, this could explain why most of the UG researchers are SNI level I and in less amount levels II or III. Regarding the distribution of researchers in each of the SNI’s knowledge fields, most of them are registered in the fields of Humanities and Behavioral Sciences and Social Sciences. These are followed by the fields of Medicine and Health Sciences, Engineering, Physics-Mathematics and Earth Sciences, Biology and Chemistry and Biotechnology and finally, Agricultural Sciences.

The UG has thematic campuses which offer a variety of academic degrees related with each of the SNI’s knowledge fields. These campuses are generally located in the Metropolitan area of Jalisco (City of Guadalajara and surrounding municipalities) but some of them are in smaller towns throughout the state. Actually, the fact that most researchers included in this study work in the Metropolitan campuses appears to indicate that the development of researchers who work in remote campuses is still a challenge.

According to the SNI’s knowledge fields, it was observed that among researchers who work at CUCS and CUCBA the scientific output of scientists in the field of MHS was the highest as measured by the average number of papers in indexed journals, number of citations and h index, followed by those registered in the fields of BC, HBS, BAS and SS.

In the evaluation of UG researchers registered in MHS and BC fields, the scientific output of researchers SNI level I, II and III was higher in MHS than in BC. In the MHS field, the average papers/citation relationship was 16/133.7, 55/473 and 94/759.6 for those with SNI level I, II and III respectively, indicating that on average researchers have (regardless of their SNI level) 8 citations per paper. Also in this field, the average h index of researchers with SNI level I is 5.4, and for those who are level II or III, it is 10.5 and 14.5 respectively. A study carried out by Romero et al. evaluated the h index of SNI scientists in the field of material sciences, finding an average h index of 4.48 for those with SNI level I, whereas it was of 6.77 in SNI level II and 12.77 in SNI level III.⁷

Currently, the number of papers and citations are essentially the basic parameters that are taken into account in the SNI’s evaluation committees. That these parameters do not reflect the quality of each paper, is mirrored by the range of the h index among the researchers of the MHS field regardless of their level. For example, there are researchers with SNI level I or II who achieve an h index of 15 while some researchers with SNI level III have an h index of 10. This shows that if the SNI’s evaluation committees use the h index as an evaluation parameter, the h index mean values obtained in this study could be taken as benchmarks for the evaluation of each SNI category.

We propose an h index of 5 to achieve the SNI level I, and for levels II and III an h index of 10 and 14-15 respectively. With this approach, the h index would then encourage researchers to produce more high quality papers instead of many papers with little impact in the international literature.

The average number of papers among the researchers with SNI level III from the UG (MHS field), capital researchers, and Emeritus scientists followed an upward increase: 94, 140.2 and 161.5 papers, respectively. Nevertheless, this pattern was not observed with their average of citations and h index. For example, the h index of capital researchers with SNI level III ranged from 11 to 38 while for Emeritus scientists and researchers with SNI level III from the UG (MHS field) ranged from 16 to 25 and 10 to 21, respectively. Again, we assume that the number of papers and citations leads to bias in the evaluation process, which could be avoided if only the h index is used as a framework which integrates both qualitative and quantitative aspects.⁵

Thus, in Mexico, the SNI could expect an h index of at least 15 for national researchers with SNI level III, while for the Emeritus scientists an h index of 20-25 should be required.

Hirsch indicates that an h index of 20 characterizes a successful researcher after 20 years of scientific work while an h index of 40 would describe scientists who have won a Nobel Prize or are leaders at universities or laboratories. Finally, an h index of 60 would characterize exceptional researchers⁴ or those who work in consolidated research groups whose outputs impact in quality and quantity. In these situations, it is even possible to find researchers who achieve an h index up to 100.

Although bibliometric databases as Scopus and Google Scholar differ on how they measure papers, citations and h index reported for each researcher,^8,9 it has been concluded that despite their differences and limitations, these are effective tools for researchers in Health Sciences.9 Nevertheless, such limitations could be overcome through the recording of papers and citations of each researcher in the course of their working lives.

The h index is increasingly being used as a parameter of quantity and quality of scientific output. It also is considered for admission and promotion of researchers within research groups, scientific associations or medical schools at an international level. For example, researchers from the Brazilian Academy of Sciences in the categories of Biomedical, Health and Chemical sciences have an average h index of 23, 20 and 19 respectively.¹⁰ In Mexico, this has not been implemented. However, the use of the h index in conjunction with the 10-index from Google Scholar (which indicates the number of papers an author has published that have at least ten citations) has been suggested to be used as a requirement for admission to medical associations like the National Academy of Medicine and/or to the Sistema Nacional de Investigadores in Mexico.¹¹

One outstanding issue regarding the evaluation of the scientific output among the members of the National Academy of Medicine is the qualitative aspect. This parameter is often vague and subjective, thus leading to injustice or dissatisfaction in the career of researchers, which in turn favors simulation and delusion behaviors. More openness and analysis are required to satisfactorily resolve these issues for the future generations of researchers.

References

1. García-Pachón E, Padilla-Navas I. El factor de impacto y el índice h de las revistas biomédicas españolas. Med Clin (Barc). 2013.
2. Editorial. Not-so-deep impact. Research assessment rests too heavily on the inflated status of the impact factor.Nature.2005;435:1003–1004.
3. www.conacyt.gob.mx/sni/paginas/default.aspx
4. Hirsch JE. An index to quantify an individual’s scientific research output. PNAS. 2005;102:16569-16572.
5. ostas R, Bordons M. The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level. J Informetrics. 2007;1: 193-203.5.
6. http://www.udg.mx/historia
7. omero AH, García A, Kiwi M. Evaluation of the scientific impact, productivity and biological age based upon the h-index in three Latin American countries: the materials science case. Ann Phys (Berlin). 2009; 18:198-205.
8. inasny B, Hartemink AE, McBratney A, Hang H. Citations and the h index of soil researchers and journals in the Web of Science, Scopus, and Google Scholar.Peer J. 2013;1: e183.
9. Cabezas-Clavijo A, Delgado-López-Cózar E. Google Scholar e índice h en biomedicina: la popularización de la evaluación bibliométrica. Med Intensiva. 2013;37:343-354.
10. ellner AWA, Ponciano LCMO. H-index in the Brazilian Academy of Sciences – comments and concerns. An Acad Bras Cienc. 2008;80:771-781.
11. Rivera H. El índice h: criterio necesario en la evaluación de investigadores. Rev Med Inst Mex Seguro Soc. 2011;49:123-124.

Enlaces refback

• No hay ningún enlace refback.