Reflections on research assessment #2: Emanuel Kulczycki on ”Essential research is published in all languages”
Is social sciences and humanities scholarship well served by the current and nascent academic reward mechanisms? And if not, where our options and power lie to change this for the better? Recognized as crucial for the future well-being and flourishing of humanistic scholarship, such questions have been repeatedly raised and discussed on DARIAH Open both in opinion pieces, in declaring commitments or policy reflections. In support of the ongoing European research assessment reform, we are extending this ongoing thread of discussion and inviting expert perspectives from the social sciences and humanities fields to further explore the issues and opportunities our research domain faces in the brave new world of research assessment. We hope in this way to spark a discussion in the arts and humanities on what we can hope to gain, and what opportunities we have to move forward.
In the second post of the series, Emanuel Kulczycki, head of the Scholarly Communication Research Group and policy advisor to the Ministry of Education and Science in Poland shares insights on the essential multilingual nature of Social Sciences and Humanities research and argues that for the reform to be successful, rehabilitating the notion of multilingualism and appropriately rewarding contributions in all languages is an absolute precondition.
In past discussions on the reform of the European research assessment system, the recognition of multilingualism was the most notable and serious omission. Fortunately, the final version of the Agreement of Reforming Research Assessment (20 July 2022) by the European Commission has taken into account language as a fundamental dimension of the scientific process and the research assessment.
In the first core commitment, the agreement emphasizes that research assessment should recognize valuable contributions “irrespective of the language in which they are communicated”. The third commitment adds that it is inappropriate to assess outputs based on “metrics relating to publication venue, format or language”. Now, it’s time to step forward and show what the academic community, policymakers, funding agencies, and research organizations can do to value scientific contributions in all languages.
Why is language as a dimension of research assessment so essential?
We must remember that almost half of the Europeans cannot speak any foreign language well enough to hold a conversation. Only a little more than one-third are able to do so in English (the most widely spoken foreign language). Thus, if research is communicated exclusively in English, it is inherently limited.
It is essential for research in social sciences and humanities. Scholars from these fields study societies and describe them using their own words. Hence, local languages are needed to express the entire cultural and historical heritage and research context. In many so-called STEM disciplines like physics or chemistry, research is often less context-specific, and it can be expressed in fewer words with figures, charts, and formulas. In social science and humanities, however, language is not a neutral medium for conveying research results: it is an essential part of the scientific process and reproducing various cultural practices and ways of thinking about social challenges.
How to fill in the term “language” used in the Agreement with the content?
Taking into account “language” in research assessment means, among others, recognizing scholarly publications in any language, promoting language diversity, and protecting national infrastructures for publishing locally relevant research.
Researchers, especially in social sciences and humanities, publish not only in their national (local) language but also—at the same time—in other languages, including English. Therefore, multilingual publishing is not publishing in a different language than English. I believe “wording” is essential. Multilingual publishing is publishing and communicating in all languages and not just in any one language, such as English or a local language. Therefore, we must acknowledge that good quality research can be published in any language and that researchers publish in various languages at the same time. Thus, research assessment cannot be reduced in any way to a single language. Scholarly communication is multilingual by its very nature.
What is a call for multilingualism in research assessment?
Instead of prioritizing local languages over English, it is a call to recognize that the quality of research does not depend on whether the research is communicated in English. Multilingualism in scholarly communication is an ongoing practice, and it is time to fully recognize and reward this priceless practice through research assessment.
Not all local language publications need to be translated into English, and not all publications in English need to be translated into other languages. Machine translation is not a substitute for publishing research in multiple languages. Instead we must stimulate and encourage multilingual publishing, communication, and dissemination practices.
However, machine translation might be one of the ways to make scientific papers accessible to readers in many languages. If we make scholarly publications really open, we can use machine translation to help scholars and other users to access papers in their own language. Nevertheless, just translating research publications to different languages does not make the contents understandable to various audiences. Therefore, we need to promote original publishing and dissemination of research results in different languages and formats. This also develops the scientific terminology, which is the backbone of translation and critical discussion of research results.
Moreover, it is crucial to improve machine translation without the mediation of English. Thus, if a publication is published in Ukrainian, we need to translate it directly into another language, e.g., Polish. I emphasize this because the current solutions do not allow me to read, for example, Turkish publications in Polish translation. First, to have a good translation, I have to translate them into English, and only then into Polish. Machine translation from Turkish to Polish is, put it mildly, very imperfect.
Is multilingualism an actual and ongoing practice in social sciences and humanities?
In the first half of 2022, Handbook on Research Assessment in the Social Sciences (co-edited by Tim C.E. Engels and myself) was published. This handbook, together with excellent authors, has provided a comprehensive overview of current developments, issues, and good practices regarding assessment in social science research. We have paid particular attention to the challenges in evaluation policies in the social sciences and the specificities of publishing in the area.
In the chapter entitled “Multilingualism of social sciences”, Tim C.E. Engels, Janne Pölönen, and myself provided empirical evidence that publication practices in social sciences are multilingual. It means that researchers in these fields regularly publish in more than one language, and a substantial number of scholars do not publish in English. We investigated, based on current national research information systems (CRIS), publication practices without language biases caused by low coverage of non-English publications in international citation indexes like Web of Science Core Collections or Scopus.
Overall, 54.2% of the social scientist from the Czech Republic, Denmark, Finland, Flanders (Belgium), Norway, Poland, and Slovenia published scientific articles in more than one language in 2013-2015. As the figure below show, however, considerable differences were observed between countries, as the share ranged from 69% in Slovenia to 34% in Flanders. Nevertheless, the practice of multilingual publishing is without a doubt a regular practice of scholarly communication
What can we do to support multilingual publishing and its proper assessment?
In reforming research evaluation, it cannot be assumed (especially in the case of the humanities and social sciences) that only publications in English are worth reading, reporting, and evaluating.
Taking into account the practical consequences of multilingual research practices presented in the chapter “Multilingualism of social sciences”, we provided the following implications for research assessment:
- Following the DORA declaration, assess research quality based on content, not language of publication.
- Make language and language biases into consideration when planning evaluation, assessment or funding procedures.
- Publications in all languages need to be taken into account.
- When enlisting evaluators, ensure that they have the required language skills to assess research outputs in relevant languages.
- Consider not only peer-reviewed publications but also publications aimed at professional and general audiences.
- When using metrics, pay attention to limited representation of research output in different languages in the standard international data sources, such as the Web of Science and Scopus, and related indicators.
- Develop and rely on comprehensive publication data that cover both peer-reviewed and other outputs, regardless of language.
These implications grow out of the values promoted by the Helsinki Initiative on Multilingualism in Scholarly Communication, launched in 2019. The Agreement of Reforming Research Assessment lists this initiative in Annex 4 as a tool to consider for improving research assessment.
The Helsinki Initiative aims to promote the language diversity and value of multilingualism in scholarly communication. Improving research assessment is one of its three main goals:
- Support the dissemination of research results for the full benefit of the society.
- Protect national infrastructures for publishing locally relevant research.
- Promote language diversity in research assessment, evaluation, and funding systems.
It is vital to describe, publish and talk in various languages if we want to reach different audiences, to show that nature, engineering, health, society and culture of most immediate relevance and importance to them are analyzed and understood in their own language. English is not enough. The reform of research assessment should not be used to produce a new Tower of Babel. The aim is to recognize that researchers already publish in many languages.
Emanuel Kulczycki – Professor of Scholarly Communication at the Adam Mickiewicz University in Poznań and science policy advisor. The chair of the European Network for Research Evaluation in the Social Sciences and the Humanities (2018-2020) and a co-founder of the Helsinki Initiative on Multilingualism in Scholarly Communication. Co-editor of the Handbook on Research Assessment in the Social Sciences (2022). More information: emanuelkulczycki.com