Skip to main content
SpringerLink
Log in

Constructing a synthetic indicator of research activity

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

There are increasing demands on universities to operate transparently with regard how resources are being used and targets met, putting them under growing pressure to clarify their position relative to other universities at national and international level. But there are several challenges associated with establishing their relative positioning. The first issue is concerned with measurement or how to capture data that are relevant, pertinent and fit-for-purpose. Second, universities must obtain an overall indicator or a means of ordering that helps synthesize the different indicators; and, third, they must decide how to weight them. Those university rankings that attempt to address these questions are met with a degree of criticism for being subjective or inconsistent in their quantification of the indicators. The aim of the present work is to develop a procedure for synthesizing all of the indicators relating to the objective measurement of university R&D and innovation into a single or summary concept. In other words, to establish a procedure that does not require subjective criteria and that can be applied for both absolute and relativized indicators. This approach makes a dual contribution. First, a specific application, in this particular case for the case of the Spanish university system, will be created, to obtain a synthetic indicator for research activity. This will enable us to achieve an R&D and innovation ranking for Spanish universities. Second, the work makes a methodological contribution by using a new technique for synthesizing this type of indicator, namely Partial Least Squares (PLS).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. The missing values are imputated using the scores predicted by a series of regressions in which each incomplete variable is regressed on the remaining variables for a given case. In the next stage (maximization), the entire set of imputated data is subjected to estimation via the maximum likelihood method. Each of these two steps is repeated in sequence until a stable solution is achieved.

References

  • Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A., & Sapir, A. (2010). The governance and performance of universities: Evidence from Europe and the US. Economic Policy, 25(61), 7–59.

    Article  Google Scholar 

  • Altbach, P. G., Reisberg, L., & Rumbley, L. E. (2010). Tracking a global academic revolution. Change: The Magazine of Higher Education, 42(2), 30–39.

    Article  Google Scholar 

  • Bagozzi, R. P. (1980). Causal models in marketing. New York: Wiley.

    Google Scholar 

  • Bentler, P. M. (1980). Multivariate analysis with latent variables: Causal models. Annual Review of Psychology, 31, 419–456.

    Article  Google Scholar 

  • Buela-Casal, G., Bermúdez-Sánchez, M. P., Sierra, J. C., Quevedo-Blasco, R., & Castro, A. (2010). Ranking de 2009 en investigación de las universidades públicas españolas. Psicothema, 22(2), 171–179.

    Google Scholar 

  • Buela-Casal, G., Gutiérrez-Martínez, O., Bermúdez-Sánchez, M. P., & Vadillo-Muñoz, O. (2007). Comparative study of international academic rankings of universities. Scientometrics, 71(3), 349–365.

    Article  Google Scholar 

  • Carayannis, E. G., & Campbell, D. F. J. (2011). Open innovation diplomacy and a 21st century fractal research, education and innovation (FREIE) ecosystem: Building on the quadruple and quintuple helix innovation concepts and the “Mode 3” knowledge production system. Journal of the Knowledge Economy, 2(3), 257–326.

    Article  Google Scholar 

  • Chen, K., & Liao, P. (2012). A comparative study on world university rankings: A bibliometric survey. Scientometrics, 92(1), 89–103.

    Article  Google Scholar 

  • Chin, W. W. (1998a). Issues and opinion on structural equation modelling. MIS Quarterly, 22(1), 8–15.

    MathSciNet  Google Scholar 

  • Chin, W. W. (1998b). The partial least squares approach to structural equation modelling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295–336). Mahwah: Lawrence Erlbaum Associates, Publisher.

    Google Scholar 

  • Del Barrio-García, S. & Luque-Martínez, T. (2012). Análisis de ecuaciones estructurales. In Luque-Martínez, T. Técnicas de análisis de datos en investigación de mercados. Madrid. Ed. Pirámide.

  • Docampo, D. (2011). On using the Shanghai ranking to assess the research performance of university systems. Scientometrics, 86(1), 77–92.

    Article  Google Scholar 

  • Docampo, D. (2012). Adjusted sum of institutional scores as an indicator of the presence of university systems in the ARWU ranking. Scientometrics, 90(1), 701–713.

    Article  Google Scholar 

  • Docampo, D., Herrera, F., Luque-Martínez, T., & Torres-Salinas, D. (2012). Efecto de la agregación de universidades españolas en el ranking de Shanghai (ARWU): caso de las comunidades autónomas y los campus de excelencia”. El profesional de la información, 21(4), 428–432.

    Article  Google Scholar 

  • Etzkowitz, H., & Leydesdorff, L. (1998). The endless transition: A ‘‘triple helix’’ of university–industry–government relations. Minerva, 36, 203–208.

    Article  Google Scholar 

  • Filippo, D., Sanz-Casado, E., Urbano Salido, C., Ardanuy, J., & Gómez-Caridad, J. (2011). El papel de las bases de datos institucionales en el análisis de la actividad científica de las universidades. Revista Española de Documentación Científica, 34(2), 165–189.

    Article  Google Scholar 

  • Fornell, C., & Cha, J. (1994). Partial least squares. In Richard P. Bagozzi (Ed.), Advanced methods of marketing research (pp. 52–78). Cambridge MA: Blackwell.

    Google Scholar 

  • Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50.

    Article  Google Scholar 

  • García, J. A., Rodríguez-Sánchez, R., Fdez-Valdivia, J., Robinson-García, N., & Torres-Salinas, D. (2012). Mapping academic institutions according to their journal publication profile: Spanish universities as a case study. Journal of the American Society for Information Science and Technology, 63(11), 2328–2340.

    Article  Google Scholar 

  • Hazelkorn, E. (2011). Rankings and the reshaping of higher education. New York: Palgrave Macmillan.

    Book  Google Scholar 

  • Ishikawa, M. (2009). University rankings, global models, and emerging hegemony critical analysis from Japan. Journal of Studies in International Education, 13(2), 159–173.

    Article  Google Scholar 

  • Jiménez-Contreras, E., Robinson-García, N., & Cabezas-Clavijo, A. (2011). Productividad e impacto de los investigadores españoles: umbrales de referencia por áreas científicas. Revista Española de Documentación Científica, 34(4), 505–526.

    Article  Google Scholar 

  • Jöreskog, K. G. (1973). Analysis of covariance structures. In P. R. Krishnaiah (Ed.), Multivariate Analysis (Vol. III). New York: Academic Press.

    Google Scholar 

  • Kline, R. B. (2011). Principles and practice of structural equation modelling (3a ed.). New York: The Guilford Press.

    MATH  Google Scholar 

  • Li, F., Yi, Y., Guo, X., & Qi, W. (2012). Performance evaluation of research universities in Mainland China, Hong Kong and Taiwan: Based on a two-dimensional approach. Scientometrics, 90, 531–542. doi:10.1007/s11192-011-0544-1.

    Article  Google Scholar 

  • Liu, N. C., & Cheng, Y. (2005). Academic ranking of world universities. methodologies and problems. Higher Education in Europe, 30(2), 127–136.

    Article  Google Scholar 

  • Lohmöller, J. B. (1989). Latent variable path modeling with partial least squares. Heidelberg: Physica-Verlag.

    Book  MATH  Google Scholar 

  • Luque-Martínez, T. (2013). La actividad investigadora de la universidad española en la primera década del siglo XXI: la importancia del tamaño de la universidad. Revista Española de Documentacion Cientifica, 36(4), 1–15. doi:10.3989/redc.2013.4.1046.

    Google Scholar 

  • Luque-Martínez, T. (2015a). Actividad investigadora y contexto económico. El caso de las universidades públicas españolas/Research activity and economic context: The case of Spanish public universities. Revista Española de Documentacion Cientifica, 38(1). doi:10.3989/redc.2015.1.1135.

  • Luque-Martínez, T. (2015b). Horizon 2031. The University of Granada in Light of its V Century. Reflections on the Future of the University. Editorial Universidad de Granada. Granada (España). http://biotic.ugr.es/pages/horizon-2031.

  • Luque-Martínez, T., & Del Barrio-García, S. (2006). Plan estratégico de la Universidad de Granada. Editorial UGR: Granada.

    Google Scholar 

  • Luque-Martínez, T., & Del Barrio-García, S. (2009). Modelling university image: The teaching staff viewpoint. Public Relations Review, 35, 325–327.

    Article  Google Scholar 

  • Luque-Martínez, T., & Doña-Toledo, L. (2013). What do graduates think? An analysis of intention to repeat the same studies and university. Journal of Marketing for Higher Education, 23(1), 62–89.

    Article  Google Scholar 

  • Marginson, S., van der Wende, M. (2007). Globalization and Higher Education. OCDE Education Working Papers No. 8, OCDE, Paris. doi:10.1787/173831738240.

  • Ringle, C. M., Wende, S., & Becker, J.-M. (2015). “SmartPLS 3.” Boenningstedt: SmartPLS GmbH, http://www.smartpls.com.

  • Robinson-García, N. (2014). Classifying and visualizing the disciplinary focus of universities. The invisible factor of university ranking. Doctoral thesis. Universidad de Granada.

  • Sanz-Casado, E. (2012). Lanzamiento del Observatorio IUNE, una herramienta para el seguimiento de la actividad científica de las universidades españolas. Revista Española de Documentación Científica, 35(3), 503–505.

    Google Scholar 

  • Sanz-Casado, E., García-Zorita, C., Serrano-López, A. E., Efraín-García, P., & De Filippo, D. (2013). Rankings nacionales elaborados a partir de múltiples indicadores frente a los de índices sintéticos. Revista Española de Documentación Científica, 36(3), 1–18.

    Article  Google Scholar 

  • Sun, Y., & Liu, F. (2012). Measuring international trade-related technology spillover: a composite approach of network analysis and information theory. Scientometrics,. doi:10.1007/s11192-012-0860-0.

    Google Scholar 

  • Torres-Salinas, D., & Jiménez-Contreras, E. (2012). Hacia las unidades de bibliometría en las universidades: modelo y funciones. Revista Española de Documentación Científica, 35(3), 469–480.

    Article  Google Scholar 

  • Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C. M., Tijssen, R. J. W., van Eck, N. J., et al. (2012). The Leiden ranking 2011-2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology, 63(12), 2419–2432.

    Article  Google Scholar 

  • Wold, H. (1982). System under indirect observation using PLS. In C. Fornell (Ed.), A second generation of multivariate analysis (pp. 325–347). New York: Praeger Publishers.

    Google Scholar 

  • Zhu, J., Hassan, S., Mirza, H., & Xie, Q. (2014). Measuring recent research performance for Chinese universities using bibliometric methods. Scientometrics, 101, 429–443.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Granada, Granada, Spain

    Teodoro Luque-Martínez & Salvador del Barrio-García

Authors
  1. Teodoro Luque-Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salvador del Barrio-García
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Teodoro Luque-Martínez.

Appendix

Appendix

See Table 4.

Table 4 Description of the indicators of research activity
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luque-Martínez, T., del Barrio-García, S. Constructing a synthetic indicator of research activity. Scientometrics 108, 1049–1064 (2016). https://doi.org/10.1007/s11192-016-2037-8

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-2037-8

Keywords

Search

Navigation