Birkbeck team devises new method to measure research impact
Index provides fresh approach to take account of factors such as 'influence' in assessing impact.
Researchers from Birkbeck’s Department of Computer Science and Information Systems have proposed an innovative new index to measure the impact of research.
The team - Professor Mark Levene, Professor Trevor Fenner and Dr. Martyn Harris from Birkbeck and Professor Judit Bar-Ilan from the department of Information Science at Bar-Ilan University – has identified the χ-index (chi-index) as a more informative way to indicate academic performance.
Debate about quality-versus-quantity in evaluating effectiveness has continued for a number of years. Many rely on the use of the “h-index” which was proposed more than a decade ago and plots the number of publications (quantity) for a researcher against the total number of citations to these publications (quality). For example, if a researcher’s h-index is 10, then he or she has published at least 10 papers each with at least 10 citations.
However, in a paper published by PLOS ONE, the team has put forward its own measure, the χ-index, comparing its effectiveness with that of the h-index.
This new index takes into account other factors, such as the way some researchers may tend to be considered influential even with only a few highly-cited publications, while others may tend to be prolific, with numerous collaborators and publications but fewer citations.
Their paper, entitled A novel bibliometric index with a simple geometric interpretation, gives an example using three researchers: one with a single publication with 100 citations; a second with 10 publications each having 10 citations; and a third with 100 publications each having a single citation. The first (influential) and third (prolific) researchers have an h-index of 1 while the second has an h-index of 10. However, using the new measure, each of these examples would have an index of 10
The team carried out a comparison of the different measures on two data sets, a Google Scholar data set of over 34,000 researchers across all disciplines and a much smaller data set of 99 Nobel Prize winners. For the first, the χ-index was significantly larger than the h-index for around 28% of the researchers, while for the Nobel Prize-winners the χ-index was significantly larger than the h-index for just over 62% of the laureates.
Professor Levene said: “We believe that the ability of the χ-index to distinguish between influential and prolific may lead to a finer ranking of researchers than allowed by the h-index. As metrics such as the h-index are increasingly being used to evaluate researchers, it is important that new alternatives such as the χ-index are put forward.”
Further Information
- A novel bibliometric index with a simple geometric interpretation
- Department of Computer Science and Information Systems
- Professor Mark Levene