Why the new Recognition & Rewards actually boosts excellent science
During the last few weeks, several opinion pieces have appeared questioning the new Recognition & Rewards (R&R) and Open Science in Dutch academia. On July 13, the TU/e Cursor published interviews with professors who question the usefulness of a new vision on Recognition & Rewards. A day later, on July 14, the chairman of the board of NWO compared science to top sport, with an emphasis on sacrifice and top performance, a line of thinking that fits the traditional way of Recognition & Rewards in academia. On July 19, an opinion piece was published by 171 university (head) teachers and professors, this time on ScienceGuide.nl questioning again the new vision of Recognition & Rewards. These articles, all published within a week, show that as the new Recognition & Rewards gains traction within universities, established scholars are questioning its usefulness and effectiveness. Like others before us, we would like to respond.
We, the signatories, do not represent the views of all Open Science Community members, just the views of the individual signatories. However, some of the signatories are active in various Open Science Communities based on the belief, in keeping with the late Jon Tennant’s slogan, that “Open Science is just science done right.” They see Open Science as better science. We do not consider this a conflict of interest, but in the context of transparency (see the Dutch Code of Conduct on Scientific Integrity) we mention it here so that the reader can form their own opinion.
The criticism surrounding Recognition & Rewards is based on a number of points:
- science comes before everything else;
- performance should be measured against existing metrics such as the Journal Impact Factor;
- Recognition & Rewards jeopardizes our top position in international science;
- open science is political and therefore should not shape how we practice science.
Science comes before everything else
The current status quo is that a “good” scientist makes personal sacrifices when doing science. This status quo encourages unhealthy working hours and workloads resulting in a high risk of mental problems such as depression and burnout. Quantity and quality are often mixed up in this discussion. For example, it is considered important that researchers publish a lot, but at the same time publish large comprehensive studies in prestigious journals. But this selection on the amount of output of a researcher makes the position of scientists who cannot or do not want to cope with this unhealthy climate extra difficult, such as researchers with care responsibilities. Many of them, mostly women, therefore decide to leave academia, resulting in a loss of many talented scientists. Currently, selection is not based on the quality of the research, but on how much the researcher is willing to work overtime. Moreover, this perspective does not do justice to the role of scientists as teachers and educators.
Performance should be tested based on existing metrics such as the Journal Impact Factor
The Journal Impact Factor and other metrics that have been used for years to evaluate academics often come up among scientists who resent the claimed immeasurability of the new Recognition & Rewards. Prestigious journals like Nature and Science are known for their high Journal Impact Factor. Even now, a Nature article can boost a scientific career because of the prestige involved. At the same time, the Journal Impact Factor says little to nothing about individual articles (7), let alone individual researchers. Even Nature says that the Journal Impact Factor cannot be used for evaluation of researchers. The evidence behind the Journal Impact Factor also shows that the impact factor does not equal excellent science, nor does it equal the number of citations or the quality of published articles. As far as the Journal Impact Factor predicts scientific quality, publications in higher Journal Impact Factor journals are on average of lower quality (Prestigious Science Journals Struggle to Reach even Average Reliability and Stagnation and Scientific Incentives). If we do want to quantify journal quality, then we should investigate supported standards such as pre-registration and standards for reporting.
Additional problems are that these prestigious journals are often not interested in replication studies. The suggestion that scientific breakthroughs are primarily in Nature, Science and Cell cannot be supported with literature.
A third problem is that the focus on the Journal Impact Factor and prestige of journals leads to exorbitant prices for open access publishing, which excludes many researchers who value applying OS principles (or are required to do so by their grantor), and/or have limited funding.
Recognition & Rewards jeopardizes our top position in international science
A frequently mentioned criticism is the notion that Recognition & Rewards jeopardizes the top position of Dutch science in international rankings. The authors of the ScienceGuide article refer to the NIH which they say focuses exclusively on excellence. Yet, the NIH only allows researchers to cite articles from previous NIH grants if they are open access. Additionally, the Journal Impact Factor plays no role at all at the NIH. Furthermore, the Matthew effect creates additional randomness that maintains the current status quo inequity. Precisely by applying the new Recognition & Rewards within Dutch science, our experience will diversify, both in content (science, education, public outreach) and in the research itself.
The global trend towards Open Science (see also the UNESCO declaration) and the issues in global science will eventually lead to the adoption of the new Dutch Recognition & Rewards practices in other countries. To date, more than 2000 organizations and more than 17000 individual researchers have signed the DORA (Declaration on Research Assessment) declaration, including major international players such as the European Research Council and prominent journals such as Nature. How nice would it be if in 10 years we could say that the balanced way we now assess scientists in international science started in the Netherlands?
Open Science is political and therefore should not shape how we practice science
The understanding that science is inevitably partly political, and the choice of openness or closedness and evaluation based on Journal Impact Factor or other criteria is not based on conclusively established objective optima, is one of the foundations of Open Science and DORA. Since this understanding has become commonplace, it is alienating to read that the status quo is presented as if it was ever instituted because it is the best approach. This is all the more true since we are all familiar with the issues in science currently regarding reproducibility of science. In most cases, this is not necessarily traceable to fraud, although in a few cases it is. The biggest cause lies in the poor description of methodologies and the invisible degrees of freedom scientists have, falling (often unknowingly) into practices such as “p-hacking“. Open Science, a major spearhead of the new Recognition & Rewards, ensures more transparency in the design, execution and reporting of research. And it is precisely this that leads to better research and thus the excellence that those professors consider very important.
Moreover, Open Science also aims at collaborating with other researchers, which means that peer review takes place at a very early stage based on the actual research, and not the polished story in the later journal article. Open Science is much more than Open Access, when scientific articles are available for free and/or with an open license to researchers and other interested parties inside and outside the university. Other important aspects of Open Science can increase the transparency and therefore the quality of research by making available the data (open data), code (open software), and reporting standards on which research is based. The choice for Open Science has already been made by the universities, VSNU, and KNAW, and is embedded in the transparency principle in the Dutch Code of Conduct on Scientific Integrity. Even if this is seen as a political choice, it is one that forms the framework for acting with integrity for Dutch scientists.
Towards a new standard
We argue that the new Recognition & Rewards is absolutely the right way forward. This is not to say that we should abandon everything of the “old way”. Science is based on change and the activities and outputs we valued before (performing decent research, publishing articles) still matter, but are no longer the only important activities and outputs. It is in our profession to alter our views based on new insights. These new insights now show us that change must come. To give all researchers a fair chance, regardless of their gender, skin color, background, origin and personal situation. Excellence cannot be measured by a few arbitrary metrics, but needs a broader view.
We recognize that this takes time, and appreciate the discussion that is currently taking place about this. There is undoubtedly still much to be gained by, for example, better training of reviewers at NWO to assess narrative CVs. Nevertheless, we believe that this new system, although not yet perfect, gives space to all science-related output, and is a step forward and science. This will ultimately also benefit the position of Dutch scientists in the international playing field.
Signed by (updated on september 6th):
If you would like to support this message, please sign here: https://forms.gle/TodrxDe3NGoN2m6Z9
- Dr. Rianne Fijten, Maastro Clinic & Maastricht University, Open Science Community Maastricht
- Dr. Gjalt-Jorn Peters, Open University of the Netherlands, Open Science Community Heerlen
- Dr. Egon Willighagen, Maastricht University, Open Science Community Maastricht
- Dr. Anita Eerland, Utrecht University, Open Science Community Utrecht
- Dr. Markus Konkol, ITC, University of Twente, Open Science Community Twente
- Dr. Dennie Hebels, Maastricht University, Open Science Community Maastricht
- Dr. Vera E. Heininga, University of Groningen, Open Science Community Groningen
- Dr. Maurits Masselink, Radboudumc, Open Science Community Groningen
- Prof. Dr. Raul Zurita-Milla, University of Twente, Open Science Community Twente
- Dr. Jonas Sundberg, Technical University of Denmark
- Mehmet Necip Tunc (PhD Candidate), Tilburg University
- Didi Lamers (PhD), Radboud University Nijmegen
- Hanne Oberman (MSc), Utrecht University, Open Science Community Utrecht
- Dr. Maikel Waardenburg (UD), Utrecht University
- Erik Roelofs, Maastricht University
- Marieke Schreuder (PhD student), UMCG, Open Science Community Groningen
- Dr. Jeanette Mostert, Radboudumc Nijmegen, Open Science Community Nijmegen
- Dr. Alie de Boer, Maastricht University
- Fionneke Bos (Postdoctoral researcher), University Medical Center Groningen
- Naomi Nota (PhD), Radboud Universiteit, Max Planck Instituut voor Psycholinguïstiek
- Leonard Wee (Docent / Assistant Professor), Maastricht University
- Patricia Romero Verdugo, Radboud University; Radboud University Medical Centre
- Stefan Frank (UHD), Radboud Universiteit
- Joanna Rutkowska, Radboud University
- Dr. Sander W. van der Laan, University Medical Center Utrecht, Open Science Community Utrecht
- Dr. Eva Poort, Max Planck Instituut voor Psycholinguïstiek, Open Science Community Nijmegen
- Prof. dr. Arjan Bos, Open Universiteit
- Dr. Sander Bosch, VU Amsterdam, Open Science Community Amsterdam
- Bram Zandbelt, Nederlandse Spoorwegen
- Frederik D. Weber, National institute for Neuroscience, Amsterdam & Donders Institute/ Radboudumc, Nijmegen, Open Science Community Nijmegen
- Kasper Gossink-Melenhorst (MSc), Responsible Research Assessment, NWO
- M. Dingemanse (Associate Professor), Radboud University, Open Science Community Nijmegen
- Dr. Christopher McCrum, Maastricht University
- Dr. Dominique Maciejewski, Radboud University Nijmegen
- Dr. Ir. Bram van Es, UMCU
- Annika Tensi (PhD Candidate), Wageningen University & Research, Open Science Community Wageningen
- Rowan Sommers, MSc., Max Planck Instituut voor Psycholinguïstiek
- Lino von Klipstein, MSc., University Medical Center Groningen
- Hajar Hasannejadasl, Maastricht University
- Ruud Houben, Maastricht University
- Dr. Karin Sanders, Utrecht University, Open Science Community Utrecht
- Dr. Surya Gayet, Donders Institute & Utrecht University
- Anshu Ankolekar (PhD Candidate), Maastro Clinic/Maastricht University
- Dr. Loek Brinkman, UMC Utrecht, Open Science Community Utrecht
- Prof. dr. Maarten Kleinhans, Universiteit Utrecht, Open Science Community Utrecht
- Merle-Marie Pittelkow (MSc), University of Groningen, Open Science Community Groningen
- Dr. Eiko Fried, Leiden University
- Alexandra Emmendorfer (PhD Candidate), Maastricht University
- Joyce Hoek (PhD student), University of Groningen
- Dr. Thomas Nagler, Leiden University
- Dr. Dagmar Eleveld, Radboudumc Nijmegen
- Erik Bijleveld (PhD), Radboud University
- Ine Noben (PhD Candidate), University of Groningen
- Dr. Marieke van Vugt, University of Groningen, Open Science Community Groningen
- Petros Kalendralis (PhD candidate), Maastricht University Medical Centre
- Dr. Marcello Seri, University of Groningen
- Michael D. Nunez (UD / Assistant Professor), University of Amsterdam
- Dr. ir. Yann de Mey, Wageningen University & Research, Open Science Community Wageningen
- Dirk van Gorp, Radboud University, Open Science Community Nijmegen
- Dr. Julia Höhler, Wageningen University, Open Science Community Wageningen
- Justin J.J. van der Hooft (Assistant Professor), Wageningen University
- Dr. Eveline Verhulst, Wageningen University, Open Science Community Wageningen
- Shanice Janssens (PhD candidate), Maastricht University
- Tessel Galesloot (PhD), Radboud university medical center, Open Science Community Nijmegen
- Prof.dr. Liesbeth Boersma, University Maastricht
- Dr. Rink Hoekstra, Rijksuniversiteit Groningen, Open Science Community Groningen
- Prof. Dr. Tim Hahn, University of Münster
- Else Eising (PhD), Max Planck Institute for Psycholinguistics
- Prof.dr.ir. Andre Dekker, Universiteit Maastricht
- Dr. Anna E. van ‘t Veer, Leiden University, Open Science Community Leiden
- Rebecca Calcott (Postdoc), Radboud University, Radboudumc
- Peder M. Isager, Eindhoven University of Technology
- Daan Ornée (MSc), Universitair Medisch Centrum Groningen, Groningen
- Dr. Sanne Booij, UMCG
- Ineke Wessel (Phd), University of Groningen
- Dr. Zsuzsika Sjoerds, Leiden University, Open Science Community Leiden
- Mariëlle Prevoo (Open Science officer), Maastricht University Library
- Marie Stadel, University of Groningen, Open Science Community Groningen
- Dr. Antonio Schettino (Coordinator Open Science), Erasmus Universiteit Rotterdam, Open Science Community Rotterdam
- Lorenza Dall’Aglio (PhD Student), Erasmus MC, Rotterdam R.I.O.T. Science club
- Laura Dijkhuizen (PhD candidate), Utrecht University, Open Science Community Utrecht
- Dr. Maikel Waardenburg (Assistant professor), Utrecht University
- Aiara Lobo Gomes (Postdoc), Maastricht University
- Dr. Jan Rijnders, Radboud Universiteit
- Carien Hilvering (Bibliometrics & Impact Specialist), Maastricht University Library
- Prof. Bartel Van de Walle, UNU-MERIT and Maastricht University
- Dr. Roman Briker, Maastricht University, Open Science Community Maastricht
- Dr. Charlotte Vrijen, University of Groningen, Open Science Community Groningen
- Dr. Anne Looijmans (Postdoctoral researcher)
- Dr. Caspar van Lissa, Utrecht University, Open Science Community Utrecht
- Stefanos Prouskas (MSc.), Amsterdam UMC
- Dr. Marc Galland, University of Amsterdam, Open Science Community Amsterdam
- Hans de Jonge (Head of Open Science Policies), NWO
- Sophie Slaats, Max Planck Instituut voor Psycholinguïstiek
- Dr. Arkady Zgonnikov, TU Delft, Open Science Community Delft
- Prof. dr. Wim de Grip, Leiden University and Radboudumc
- Andres Felipe Ramos Padilla, University of Groningen, Open Science Community Groningen
- Rachelle Swart (PhD / Policy Officer Research), Maastro & Maastricht University
- Dr. Linda Rieswijk, Maastricht University Institute of Data Science
- Drs. Matthijs Sloep, Maastricht University, Maastricht open science community
- Salina Thijssen (MSc), Maastricht University Medical Centre
- Olmo van den Akker (MSc), Tilburg University, Open Science Community Tilburg