Abstract

The aim of the article is to create a scheme of evaluation of key performance indicators of science at universities, which can be calculated on the basis of available statistics; to offer the levels of performance that meet the world’s ideas about the world-class research universities and correspond to international practice. The results of the analysis. Our study confirmed the findings of scholars that the research performance indicators, adopted by the Government of Ukraine in “Regulations on the research university” in 2010, did not correspond to the idea of the world-class universities. Using these indicators do not help to achieve high scientific performance. For example, the number of publications in Scopus of leading European universities typically more than 20 times higher than specified in the Regulations. One of our conclusions is that to create the world-class research universities in the country we should identify the appropriate science performance indicators and to ensure their achievement rather than to provide research university status without proper level of the results and adequate support. Modern systems of the university science evaluation use a number of articles and their citations as indicators in absolute values and per unit indicators. Despite the availability of information on research and development of Ukrainian universities, there is no methodology to calculate the number of researchers in the “full time equivalent” (FTE) and the science productivity per unit indicators. It is proposed the original scheme of calculating the reduced number of researchers in accordance with the FTE approach. We generalize the experience of Ukrainian universities assessment in terms of scientific and scientific applied results to obtain by the different groups of scientists. The main procedure of this scheme is to calculate the sum of researchers in FTE, which consist of: – the total number of university lecturers and «external» part-time researchers, taken into account with the coefficient equal to 0,5; – the total number of staff researchers and the doctoral students – with the coefficient of 1; – the total number of graduate students – with the coefficient of 2/3. The proposed scheme can provide to calculate the scientific performance indicators for university assessment and ranking with rigor sufficient to determine they place among other institutions. Propositions to define the main indicators of scientific results per FTE researchers are worked out. According to our research, the main indicators per researcher in full-time equivalent should include not only the number of publications and citations in Scopus, and h-index calculations, but also the volume of funds earned by scientists in carrying out research and development by grants and projects. The total number of publications (including not indexed in Scopus and Web of Science) is a very important source, but our statistics do not ready to differentiate between them on quality. The tables show the main indicators for classical and technical universities, which are among the leaders in terms of publications in Scopus. Conclusions and directions of further researches. By using the proposed scheme of calculating the number of researchers in FTE we have received results that make it possible to rank universities according to their intensity to obtain scientific and scientific applied results. These calculations also provide an opportunity to compare the intensity of research activities in different countries. Further studies could to improve the evaluation of the number of researchers in FTE basing on an approach, which will provide an opportunity to clarify the parts of research activities for specific groups of university researchers. It is also necessary to determine how best to assess the articles in the national and international journals that are not indexed in Scopus and Web of Science, and how to make the assessment of other scientific and scientific applied results more transparent, understandable and correct. Among the problems that can be solved for the correct evaluation of university performance remain issues of incomparability and incommensurability of criteria and indicators. In our view, there should be found partial solutions and useful clarifications.

Keywords: research university, research, development, indicator, full-time equivalent, scientific result, scientific applied result

JEL Classification: O38, M49, A14.

Cite as: Porev S. & Sandyga, I. (2016). Indicators of science that are critical for creation of Ukrainian research universities. Marketing and Management of Innovations, 3, 246-262. https://doi.org/10.21272/mmi.2016.3-19

This work is licensed under a Creative Commons Attribution 4.0 International License

References

1. Porev, S. (2012). Universitet i nauka. Epistemolohiia, metodolohiia i pedahohika vyrobnytstv znan [University & Science. Epistemology, methodology and pedagogy of knowledge producnion]. Kyiv: Himdzest [in Ukrainian].
2. Postanova Kabinetu Ministriv Ukrainy «Pro zatverdzhennia Polozhennia pro doslidnitskyi universitet» [Resolution of the Cabinet of Ministers of Ukraine «On approval of the Regulation on Research University»]. from 17.02.2010  №163] (n.d.). zakon.rada.gov.ua. Retrieved from: http://zakon1.rada.gov.ua/laws/show/163-2010-%D0%BF [in Ukrainian]. 
3. Postanova Kabinetu Ministriv Ukrainy «Pro vnesennia zmin do postanovy Kabinetu Ministriv Ukrainy vid 17 liutoho 2010 №163» [Resolution of the Cabinet of Ministers of Ukraine «On making changes to the resolution of the Cabinet of Ministers from 17th February 2010 №163»] (n.d.). zakon.rada.gov.ua. Retrieved from http://zakon1.rada.gov.ua/laws/show/786-2010-%D0%BF [in Ukrainian].
4. Postanova Kabinetu Ministriv Ukrainy «Pro vnesennia zmin ta viznannia takymy, shcho vtratyly chinnist, deiakykh aktiv Kabinetu Ministriv Ukrainy vid 5 noiabria 2014 №597» [Resolution of the Cabinet of Ministers of Ukraine «On making changes and ceasing to be invalid some acts of the Cabinet of Ministers of Ukraine» from 5th November 2014 №597] (n.d.). zakon.rada.gov.ua. Retrieved from http://zakon1.rada.gov.ua/laws/show/597-2014-%D0%BF [in Ukrainian].
5. Zakon Ukrainy «Pro vyshchu osvitu» [The Law of Ukraine «On higher education»] (n.d.). zakon.rada.gov.ua.  Retrieved from: http://zakon5.rada.gov.ua/laws/show/1556-18/page2 [in Ukrainian].
6. EuroOsvita.Net. Retrieved from http://www.euroosvita.net/index.php/?category=58 [in Ukrainian].
7. Edyna informatsiina systema «Nauka v universytetakh» [Unified information system «Science in Universities»]. kis.mon.gov.ua. Retrieved from http://kis.mon.gov.ua [in Ukrainian].
8. Rauhvargers, A. (2013). Global university rankings and their impact. (Report ІІ). Brussels: European University Association [in English].
9. Saisana M.& D’Hombres, B. (2008). Higher Education Rankings: Robustness Issues and Critical Assessment. EUR 23487, Joint Research Centre. Italy, Publications Office of the European Union [in English].
10. Division for Analysis and Evaluation. (2013). University ranking list – a directory.  (Report: 013:03. № V 2012/523). University of Gothenburg [in English].
11. Altbach, Ph.G., & Salmi, J. (Eds.). (2011). The road to academic excellence: the making of world-class research universities. Washington, DC:The World Bank [in English].
12. Oleksiyenko, A. (2014). Socio-Economic Forces and the Rise of the World-Class Research University in the Post-Soviet Higher Education Space: The Case of Ukraine. European Journal of Higher Education, 1-17 [in English].
13. Hladchenko, M., de Boer H.F., & Westerheijden D.F. (2016) Establishing research universities in Ukrainian higher education: the incomplete journey of a structural reform, Journal of Higher Education Policy and Management, 38:2, 111-125 [in English].
14. European Commission. (2010). Assessing Europe’s University-Based Research. Expert Group on Assessment of University-Based Research. (RTD.C4). Luxembourg: Publications Office of the European Union [in English].
15. Van Raan, A.F.J. (2007). Bibliometric statistical properties of the 100 largest European universities: prevalent scaling rules in the science system. Centre for Science and Technology Studies. Leiden University, 1-26 [in English].
16. Labrosse, I., Ventimiglia, A., Campbel, D., Haustein, S., Cote, G., & Archambault, E. (2013). Scientific output and collaboration of European Universities. European Commission, Directorate-General for Research and Innovation, EUR 26116 EN [in English].
17. OECD. (2002). Frascati Manual Proposed Standard Practice for Surveys on Research and Experimental Development. Paris, France: OECD [in English].
18. OECD (2013), Main Science and Technology Indicators Volume 2012 Issue 2, OECD Publishing [in English].
19. Bornmann, L.& Marx, W. (2011). The h index as a research performance indicator. European Science Editing. 37(№3), 77-80 [in English].
20. NSF. (2015). The National Science Foundation Proposal and award policies and procedures guide. NSF 16-1 [in English].
21. Smart, W. (2009). The impact of the performance – based research fund on the research productivity of New Zealand universities. Social Policy Journal of New Zealand,  (34), 136-151 [in English].