Responsible Use of Metrics
When used alongside other evaluative measures, bibliometrics can be a useful tool for evaluating research. However, all bibliometrics indicators have limitations and should not be used out of context or applied without a full understanding of their intended use.
Some Pros & Cons of Bibliometrics
- Quantitative, objective and reproducible
- Easy to understand and easily updated
- Fully scalable - from individual- to country-level
- Datasets, particularly from standard databases like Web of Science (WOS), may represent only a portion of existing publications
- Most indicators are skewed and are vulnerable to manipulation by authors & publishers
- Indicators don’t necessarily mean what we think they mean (e.g. a high citation count may be the result of “negative” citations rather than an indicator of quality)
Bibliometrics Are Not the Whole Story
No single metric can provide a rounded overview of research performance so responsible use of metrics requires using multiple metrics and providing context for those metrics. It can be helpful to think of a bibliometric analysis as a story where each indicator is a plot point. Additionally, bibliometrics should not be used as the sole basis for decision-making or for evaluating the work of either an individual or group.
Many patrons ask the bibliometrics team if Google Scholar can be used in place of Web of Science (WOS) as a source of data due to its increased coverage. However, Google Scholar has limitations which affect the practicality of using it as a bibliometrics data source, and as a tool for deriving metrics such as an h-index. There are also concerns regarding the validity and accuracy of the citation data provided by Google Scholar. Due to these concerns and limitations our team generally relies on WOS and other tools for data and analysis.