A. Cecile J.W. Janssens, PhD, professor of epidemiology at the Emory University Rollins School of Public Health, and colleagues have authored a paper published in PLOS Biology that questions the validity of the recently proposed Relative Citation Ratio (RCR).

The RCR is a metric that was created as a method for measuring the influence of a scientific paper by comparing the citation rate of a paper against others in the same field, and has become an important metric in the National Institutes of Health's (NIH) grant management policy. While Janssens agrees that the metric seems attractive and intuitive, she writes that after analyzing the algorithm behind the RCR, "concerns warrant further research for reliable and valid fixes before the metric can be implemented in grant management policies."

Key among Janssens' observations were: 

• The article citation rate (the numerator) is expected to decrease for older publications and may be too unstable for recent and lowly cited publications.

• The definition of what constitutes a paper's "field," namely all its co-cited articles, is invalid. Most co-cited articles are unrelated to the article for which the RCR is calculated.

• The normalization procedure entails a major adjustment of RCR values, which suggests the two-year journal impact factors are not a suitable representation for the field citation rate (the denominator).

These factors lower the RCR for older articles, which may negatively affect the RCR of the portfolios of advanced career researchers and underestimate their scientific influence.

Janssens emphasizes the need for a rigorous analysis of high-quality data in developing a policy about performance metrics.