During the last few weeks, several opinion pieces have appeared questioning the new Recognition & Rewards (R&R) and Open Science in Dutch academia. On July 13, the TU/e Cursor published interviews with professors who question the usefulness of a new vision on Recognition & Rewards. A day later, on July 14, the chairman of the board of NWO compared science to top sport, with an emphasis on sacrifice and top performance, a line of thinking that fits the traditional way of Recognition & Rewards in academia. On July 19, an opinion piece was published by 171 university (head) teachers and professors, this time on ScienceGuide.nl questioning again the new vision of Recognition & Rewards. These articles, all published within a week, show that as the new Recognition & Rewards gains traction within universities, established scholars are questioning its usefulness and effectiveness. Like others before us, we would like to respond.

We, the signatories, do not represent the views of all Open Science Community members, just the views of the individual signatories. However, some of the signatories are active in various Open Science Communities based on the belief, in keeping with the late Jon Tennant’s slogan, that “Open Science is just science done right.” They see Open Science as better science. We do not consider this a conflict of interest, but in the context of transparency (see the Dutch Code of Conduct on Scientific Integrity) we mention it here so that the reader can form their own opinion.

The criticism surrounding Recognition & Rewards is based on a number of points:

  • science comes before everything else;
  • performance should be measured against existing metrics such as the Journal Impact Factor;
  • Recognition & Rewards jeopardizes our top position in international science;
  • open science is political and therefore should not shape how we practice science.

Science comes before everything else
The current status quo is that a “good” scientist makes personal sacrifices when doing science. This status quo encourages unhealthy working hours and workloads resulting in a high risk of mental problems such as depression and burnout. Quantity and quality are often mixed up in this discussion. For example, it is considered important that researchers publish a lot, but at the same time publish large comprehensive studies in prestigious journals. But this selection on the amount of output of a researcher makes the position of scientists who cannot or do not want to cope with this unhealthy climate extra difficult, such as researchers with care responsibilities. Many of them, mostly women, therefore decide to leave academia, resulting in a loss of many talented scientists. Currently, selection is not based on the quality of the research, but on how much the researcher is willing to work overtime. Moreover, this perspective does not do justice to the role of scientists as teachers and educators.

Performance should be tested based on existing metrics such as the Journal Impact Factor
The Journal Impact Factor and other metrics that have been used for years to evaluate academics often come up among scientists who resent the claimed immeasurability of the new Recognition & Rewards. Prestigious journals like Nature and Science are known for their high Journal Impact Factor. Even now, a Nature article can boost a scientific career because of the prestige involved. At the same time, the Journal Impact Factor says little to nothing about individual articles (7), let alone individual researchers. Even Nature says that the Journal Impact Factor cannot be used for evaluation of researchers. The evidence behind the Journal Impact Factor also shows that the impact factor does not equal excellent science, nor does it equal the number of citations or the quality of published articles. As far as the Journal Impact Factor predicts scientific quality, publications in higher Journal Impact Factor journals are on average of lower quality (Prestigious Science Journals Struggle to Reach even Average Reliability and Stagnation and Scientific Incentives). If we do want to quantify journal quality, then we should investigate supported standards such as pre-registration and standards for reporting.

Additional problems are that these prestigious journals are often not interested in replication studies. The suggestion that scientific breakthroughs are primarily in Nature, Science and Cell cannot be supported with literature.

A third problem is that the focus on the Journal Impact Factor and prestige of journals leads to exorbitant prices for open access publishing, which excludes many researchers who value applying OS principles (or are required to do so by their grantor), and/or have limited funding.

Recognition & Rewards jeopardizes our top position in international science
A frequently mentioned criticism is the notion that Recognition & Rewards jeopardizes the top position of Dutch science in international rankings. The authors of the ScienceGuide article refer to the NIH which they say focuses exclusively on excellence. Yet, the NIH only allows researchers to cite articles from previous NIH grants if they are open access. Additionally, the Journal Impact Factor plays no role at all at the NIH. Furthermore, the Matthew effect creates additional randomness that maintains the current status quo inequity. Precisely by applying the new Recognition & Rewards within Dutch science, our experience will diversify, both in content (science, education, public outreach) and in the research itself.

The global trend towards Open Science (see also the UNESCO declaration) and the issues in global science will eventually lead to the adoption of the new Dutch Recognition & Rewards practices in other countries. To date, more than 2000 organizations and more than 17000 individual researchers have signed the DORA (Declaration on Research Assessment) declaration, including major international players such as the European Research Council and prominent journals such as Nature. How nice would it be if in 10 years we could say that the balanced way we now assess scientists in international science started in the Netherlands?

Open Science is political and therefore should not shape how we practice science
The understanding that science is inevitably partly political, and the choice of openness or closedness and evaluation based on Journal Impact Factor or other criteria is not based on conclusively established objective optima, is one of the foundations of Open Science and DORA. Since this understanding has become commonplace, it is alienating to read that the status quo is presented as if it was ever instituted because it is the best approach. This is all the more true since we are all familiar with the issues in science currently regarding reproducibility of science. In most cases, this is not necessarily traceable to fraud, although in a few cases it is. The biggest cause lies in the poor description of methodologies and the invisible degrees of freedom scientists have, falling (often unknowingly) into practices such as “p-hacking“. Open Science, a major spearhead of the new Recognition & Rewards, ensures more transparency in the design, execution and reporting of research. And it is precisely this that leads to better research and thus the excellence that those professors consider very important.

Moreover, Open Science also aims at collaborating with other researchers, which means that peer review takes place at a very early stage based on the actual research, and not the polished story in the later journal article. Open Science is much more than Open Access, when scientific articles are available for free and/or with an open license to researchers and other interested parties inside and outside the university. Other important aspects of Open Science can increase the transparency and therefore the quality of research by making available the data (open data), code (open software), and reporting standards on which research is based. The choice for Open Science has already been made by the universities, VSNU, and KNAW, and is embedded in the transparency principle in the Dutch Code of Conduct on Scientific Integrity. Even if this is seen as a political choice, it is one that forms the framework for acting with integrity for Dutch scientists.

Towards a new standard
We argue that the new Recognition & Rewards is absolutely the right way forward. This is not to say that we should abandon everything of the “old way”. Science is based on change and the activities and outputs we valued before (performing decent research, publishing articles) still matter, but are no longer the only important activities and outputs. It is in our profession to alter our views based on new insights. These new insights now show us that change must come. To give all researchers a fair chance, regardless of their gender, skin color, background, origin and personal situation. Excellence cannot be measured by a few arbitrary metrics, but needs a broader view.

We recognize that this takes time, and appreciate the discussion that is currently taking place about this. There is undoubtedly still much to be gained by, for example, better training of reviewers at NWO to assess narrative CVs. Nevertheless, we believe that this new system, although not yet perfect, gives space to all science-related output, and is a step forward and science. This will ultimately also benefit the position of Dutch scientists in the international playing field.

Signed by (updated on september 6th):
If you would like to support this message, please sign here: https://forms.gle/TodrxDe3NGoN2m6Z9

  1. Dr. Rianne Fijten, Maastro Clinic & Maastricht University, Open Science Community Maastricht
  2. Dr. Gjalt-Jorn Peters, Open University of the Netherlands, Open Science Community Heerlen
  3. Dr. Egon Willighagen, Maastricht University, Open Science Community Maastricht
  4. Dr. Anita Eerland, Utrecht University, Open Science Community Utrecht
  5. Dr. Markus Konkol, ITC, University of Twente, Open Science Community Twente
  6. Dr. Dennie Hebels, Maastricht University, Open Science Community Maastricht
  7. Dr. Vera E. Heininga, University of Groningen, Open Science Community Groningen
  8. Dr. Maurits Masselink, Radboudumc, Open Science Community Groningen
  9. Prof. Dr. Raul Zurita-Milla, University of Twente, Open Science Community Twente
  10. Dr. Jonas Sundberg, Technical University of Denmark
  11. Mehmet Necip Tunc (PhD Candidate), Tilburg University
  12. Didi Lamers (PhD), Radboud University Nijmegen
  13. Hanne Oberman (MSc), Utrecht University, Open Science Community Utrecht
  14. Dr. Maikel Waardenburg (UD), Utrecht University
  15. Erik Roelofs, Maastricht University
  16. Marieke Schreuder (PhD student), UMCG, Open Science Community Groningen
  17. Dr. Jeanette Mostert, Radboudumc Nijmegen, Open Science Community Nijmegen
  18. Dr. Alie de Boer, Maastricht University
  19. Fionneke Bos (Postdoctoral researcher), University Medical Center Groningen
  20. Naomi Nota (PhD), Radboud Universiteit, Max Planck Instituut voor Psycholinguïstiek
  21. Leonard Wee (Docent / Assistant Professor), Maastricht University
  22. Patricia Romero Verdugo, Radboud University; Radboud University Medical Centre
  23. Stefan Frank (UHD), Radboud Universiteit
  24. Joanna Rutkowska, Radboud University
  25. Dr. Sander W. van der Laan, University Medical Center Utrecht, Open Science Community Utrecht
  26. Dr. Eva Poort, Max Planck Instituut voor Psycholinguïstiek, Open Science Community Nijmegen
  27. Prof. dr. Arjan Bos, Open Universiteit
  28. Dr. Sander Bosch, VU Amsterdam, Open Science Community Amsterdam
  29. Bram Zandbelt, Nederlandse Spoorwegen
  30. Frederik D. Weber, National institute for Neuroscience, Amsterdam & Donders Institute/ Radboudumc, Nijmegen, Open Science Community Nijmegen
  31. Kasper Gossink-Melenhorst (MSc), Responsible Research Assessment, NWO
  32. M. Dingemanse (Associate Professor), Radboud University, Open Science Community Nijmegen
  33. Dr. Christopher McCrum, Maastricht University
  34. Dr. Dominique Maciejewski, Radboud University Nijmegen
  35. Dr. Ir. Bram van Es, UMCU
  36. Annika Tensi (PhD Candidate), Wageningen University & Research, Open Science Community Wageningen
  37. Rowan Sommers, MSc., Max Planck Instituut voor Psycholinguïstiek
  38. Lino von Klipstein, MSc., University Medical Center Groningen
  39. Hajar Hasannejadasl, Maastricht University
  40. Ruud Houben, Maastricht University
  41. Dr. Karin Sanders, Utrecht University, Open Science Community Utrecht
  42. Dr. Surya Gayet, Donders Institute & Utrecht University
  43. Anshu Ankolekar (PhD Candidate), Maastro Clinic/Maastricht University
  44. Dr. Loek Brinkman, UMC Utrecht, Open Science Community Utrecht
  45. Prof. dr. Maarten Kleinhans, Universiteit Utrecht, Open Science Community Utrecht
  46. Merle-Marie Pittelkow (MSc), University of Groningen, Open Science Community Groningen
  47. Dr. Eiko Fried, Leiden University
  48. Alexandra Emmendorfer (PhD Candidate), Maastricht University
  49. Joyce Hoek (PhD student), University of Groningen
  50. Dr. Thomas Nagler, Leiden University
  51. Dr. Dagmar Eleveld, Radboudumc Nijmegen
  52. Erik Bijleveld (PhD), Radboud University
  53. Ine Noben (PhD Candidate), University of Groningen
  54. Dr. Marieke van Vugt, University of Groningen, Open Science Community Groningen
  55. Petros Kalendralis (PhD candidate), Maastricht University Medical Centre
  56. Dr. Marcello Seri, University of Groningen
  57. Michael D. Nunez (UD / Assistant Professor), University of Amsterdam
  58. Dr. ir. Yann de Mey, Wageningen University & Research, Open Science Community Wageningen
  59. Dirk van Gorp, Radboud University, Open Science Community Nijmegen
  60. Dr. Julia Höhler, Wageningen University, Open Science Community Wageningen
  61. Justin J.J. van der Hooft (Assistant Professor), Wageningen University
  62. Dr. Eveline Verhulst, Wageningen University, Open Science Community Wageningen
  63. Shanice Janssens (PhD candidate), Maastricht University
  64. Tessel Galesloot (PhD), Radboud university medical center, Open Science Community Nijmegen
  65. Prof.dr. Liesbeth Boersma, University Maastricht
  66. Dr. Rink Hoekstra, Rijksuniversiteit Groningen, Open Science Community Groningen
  67. Prof. Dr. Tim Hahn, University of Münster
  68. Else Eising (PhD), Max Planck Institute for Psycholinguistics
  69. Prof.dr.ir. Andre Dekker, Universiteit Maastricht
  70. Dr. Anna E. van ‘t Veer, Leiden University, Open Science Community Leiden
  71. Rebecca Calcott (Postdoc), Radboud University, Radboudumc
  72. Peder M. Isager, Eindhoven University of Technology
  73. Daan Ornée (MSc), Universitair Medisch Centrum Groningen, Groningen
  74. Dr. Sanne Booij, UMCG
  75. Ineke Wessel (Phd), University of Groningen
  76. Dr. Zsuzsika Sjoerds, Leiden University, Open Science Community Leiden
  77. Mariëlle Prevoo (Open Science officer), Maastricht University Library
  78. Marie Stadel, University of Groningen, Open Science Community Groningen
  79. Dr. Antonio Schettino (Coordinator Open Science), Erasmus Universiteit Rotterdam, Open Science Community Rotterdam
  80. Lorenza Dall’Aglio (PhD Student), Erasmus MC, Rotterdam R.I.O.T. Science club
  81. Laura Dijkhuizen (PhD candidate), Utrecht University, Open Science Community Utrecht
  82. Dr. Maikel Waardenburg (Assistant professor), Utrecht University
  83. Aiara Lobo Gomes (Postdoc), Maastricht University
  84. Dr. Jan Rijnders, Radboud Universiteit
  85. Carien Hilvering (Bibliometrics & Impact Specialist), Maastricht University Library
  86. Prof. Bartel Van de Walle, UNU-MERIT and Maastricht University
  87. Dr. Roman Briker, Maastricht University, Open Science Community Maastricht
  88. Dr. Charlotte Vrijen, University of Groningen, Open Science Community Groningen
  89. Dr. Anne Looijmans (Postdoctoral researcher)
  90. Dr. Caspar van Lissa, Utrecht University, Open Science Community Utrecht
  91. Stefanos Prouskas (MSc.), Amsterdam UMC
  92. Dr. Marc Galland, University of Amsterdam, Open Science Community Amsterdam
  93. Hans de Jonge (Head of Open Science Policies), NWO
  94. Sophie Slaats, Max Planck Instituut voor Psycholinguïstiek
  95. Dr. Arkady Zgonnikov, TU Delft, Open Science Community Delft
  96. Prof. dr. Wim de Grip, Leiden University and Radboudumc
  97. Andres Felipe Ramos Padilla, University of Groningen, Open Science Community Groningen
  98. Rachelle Swart (PhD / Policy Officer Research), Maastro & Maastricht University
  99. Dr. Linda Rieswijk, Maastricht University Institute of Data Science
  100. Drs. Matthijs Sloep, Maastricht University, Maastricht open science community
  101. Salina Thijssen (MSc), Maastricht University Medical Centre
  102. Olmo van den Akker (MSc), Tilburg University, Open Science Community Tilburg

Leave a Reply

Your email address will not be published. Required fields are marked *