The Use of Bibliometric Methods for Research Evaluation
On May 12th Prof. Yves Gingras has delivered a workshop "The Use of Bibliometric Methods for Research Evaluation" at the HSE. The event has been organized by the Research Laboratory of Science and Technology Studies, HSE Institute for Statistical Studies and Economics of Knowledge.
On May 12th Prof. Yves Gingras has delivered a workshop "The Use of Bibliometric Methods for Research Evaluation" at the HSE. The event has been organized by the Research Laboratory of Science and Technology Studies, HSE Institute for Statistical Studies and Economics of Knowledge.
Yves Gingras is Professor in the Department of History at the University of Quebec in Montreal, former Director of the Interuniversity Research Center on Science and Technology (CIRST), Scientific Director of the Observatory of Science and Technology (OST), and Canada Research Chair in History and Sociology of Science.
The speaker described the main tools of quantitative assessment of science and technology, which are based on indicators of input, output and impact. He stressed the importance of using differentiated approach when choosing a method for evaluation of scientific activity depending on the area of knowledge and aims of valuation procedure. One should take into account numerous aspects of the impact posed by scientific developments, including scientific, technological, economic, social, political, symbolic, and others.
The professor pointed out a recent awkward trend, consisting in bringing down these many aspects to a single dimension, and, accordingly, in search of a universal indicator, such as economic efficiency or bibliometric indicators based on data on the number of publications and citations. Such a unilateral approach entails negative consequences for the knowledge production and education systems.
Yves Gingras provided a brief historical overview of bibliometrics, spoke about the opportunities and main areas of application of this analytical tool, as well as basic bibliometric indicators. Currently, bibliometric methods are widely used for assessing the performance of countries, universities, etc., benchmarking, ranking of countries and scientific organizations, studying selected fields of science, mapping out of science, identification of current and emerging scientific areas, etc.
In conclusion the lecturer presented a detailed analysis of "good" and "bad" indicators. A "good" indicator is characterized by the 
relevance to the object of assessment, homogeneity of measurement, changing values in the time interval corresponding to the inertia of the object (e.g., university cannot dramatically raise or lower the rating within 2-3 years).
Well known university rankings i.e. compiled by the Shanghai University and Times journal as well as the h-index are based on composite indicators, which do not meet the above parameters. Wide application of these indicators actually gives a distorted insight on "quality" of education in a particular university and on performance of individual researchers or research organizations.
The analysis yielded the following conclusions:
- Bibliometrics is a valuable tool for analyzing the structure and dynamics of scientific knowledge, which is not always obvious to local actors;
- Indicators should take into account the specifics of each discipline;
- Bibliometrics can be useful tool for decision-making, but cannot be used as its substitution.