Thing 22: Research Impact

Researchers are often required to support grant or promotion applications with evidence of the impact of their research. In this post we will look at “traditional” metrics that may be used as a way to measure research impact.

If we assume that quality research is cited more often by other scholars, then citation counts, journal impact factors, h-index and other metrics can be considered quantitative measures of research impact.

Metrics for Measuring Research Impact

Article-Level Metrics

As the name suggests, article-level metrics refer to a variety of measures that provide insight into the reach of individual articles. Citation counts and alternative metrics such as downloads, views and media/social media mentions are examples of article-level metrics. 

Web of Science, Scopus and Google Scholar are commonly used as a source for citation counts but the results from each database may be different as these counts are dependent on the indexed publications. One of the features of each of these databases is the ability to click on through and view the articles that have cited a particular paper. 

Searching for individual articles in Web of Science, Scopus and Google Scholar will display citation counts, as shown in the following examples:   

Creating and/or maintaining Researcher/Author profiles within these databases is the most effective way of keeping track of citation counts for your publications. See the Researcher Profiles Subject Research Guide and Thing 21: Researcher IDs for more information.   

Alternative Article-Level Metrics

Complementary to the “traditional” metrics provided by citation counts, alternative article-level metrics are based on online activity, mined or gathered from online tools and social media (e.g. example tweets, mentions, shares or links). Thing 23 introduces “Altmetrics”, and how to measure them, in more detail; also see the Altmetrics Library Guide for information.

Journal-Level Metrics

Journal-level metrics aim to measure the influence of a journal. There are many journal-level metrics based on different methods of calculation and datasets. The impact factors and Scopus journal metrics are two commonly used measures or rankings of journal quality which rely on the number of citations the journal has received in Web of Science and Scopus respectively. 

Scopus Journal Metrics (based on Elsevier data) provide journal rankings in subject categories and various metrics – see the screenshot below for an example of Scopus Journal Metrics, and here for more information:

 

Clarivate’s Journal Impact Factor allows for the comparison and ranking of journals within the same discipline:

See here for more information on the Journal Impact Factor.

Author-Level Metrics

Author-level metrics – such as such as the h-index – measure the impact of individual researchers. A researcher’s h-index can be calculated manually by locating citation counts for all published papers and ranking them numerically by the times cited: a researcher has an index of h if h of their papers have been cited at least h times each. 

As mentioned above, the best way to keep track of your author metrics is to create and keep up-to-date your author profiles, for example:   

• Scopus author profile
• ResearcherID profile (Web of Science) 
  
• Google Scholar Citations profile. 

If you establish and maintain author profiles it will be very easy for you and others (e.g. grant application panels) to check your author-level metrics, or for you to link to these IDs in your grant applications. 

Considerations

• No single metric can be used as an indicator of research impact or quality. Tools that provide citation and other metrics, such as Scopus, Web of Science and Google Scholar are basing these metrics only on the publications they index. 
• Some funding bodies (such as the NHMRC) do not allow the use of a journal impact factor or h-index in grant applications – and rules change from year to year. When preparing a grant application you should read and follow the rules and instructions for the grant in question to ensure you only include accepted metrics.

Try This

View this short video ‘The research counts, not the journal!’, where Nobel Laureates take a stand against the role of impact factors in research:

Learn More

To find out more about how the University of Melbourne Library can assist you to manage, monitor, measure, and maximize your research impact click here. Further relevant articles and blogposts: 

• Anne-Wil Harzing: The four C’s of getting cited  
• MyRI: Calculating the h-index: Web of Science, Scopus or Google Scholar? 

This post was written by Peta Humphreys (Science & Engineering Faculty Librarian), Lindy Cochrane (Liaison Librarian (Research), Medicine, Dentistry and Health Sciences) and Satu Alakangas (Liaison Librarian (Research), Law).

Image: “water drop droplet liquid mirror science” by qimono via Pixabay (CC0)