Can Everything in Science Be Measured?

Giorgio Sirilli, Associate Research Director at Italy’s Research Institute on Sustainable Economic Growth of National Research Council (IRCrES), an active participant and former chair of the OECD Working Party of National Experts on Science and Technology Indicators (NESTI), and author of over 200 academic publications, gave an open lecture at HSE on January 21, 2015

Russia takes an active part in NESTI work since 1992. Delegates from Russia – Leonid Gokhberg and Konstantin Fursov – are actively involved in numerous NESTI projects on development and use of indicators for statistical measurement of science and technology.

Giorgio Sirilli highlighted the importance of providing information that is as objective and value-neutral as possible. Based on general methodological principles, statistics always follows the political agenda. If, initially, the statistical measurement of R&D was mainly focused on scientific and technological activity in order to strengthen its role in improving the national welfare and security, then today it is the very processes of innovative development, including those focused on identifying and responding to societal and global environmental challenges, that are the subject of regulation and statistical analysis.

On the un-measurable

Statistics cannot encompass the entire field of science, technology and innovation, argues Sirilli. The mass of information that is subject to systematized description and measurement is just the tip of the iceberg. Most processes that push forward scientific development and define its success in any country will remain hidden. Therefore, interpreting survey results and making decisions based on them, especially political ones, requires the utmost care. Sirilli illustrated this with an example of using bibliometric indicators for evaluating scientific productivity of research and educational institutions.

For much of the lecture, Sirilli focused on pros and cons of complex indexes making it possible to evaluate strong and weak sides of innovative systems in any particular country, and to compare different aspects of different economies.

Data visualization and the magic of numbers

The ways in which information is presented largely determine its perception by users.

Consequently, in order to ensure the widest possible audience gets an accurate picture of STI development in a country, statistics should be based on a broad arsenal of data visualization tools. At the same time, it is desirable that the figures should be presented without involving subjective evaluations.

In studying complex systems such as science, technology and innovation, Sirilli concluded, it is important to be particularly careful when selecting statistical units and interpreting the results of observations, taking into account limitations of the methods used. Statistics prepares data for decision-making, opens up new horizons and opportunities, but the figures never give us a full representation of an object or a phenomenon under analysis.