Posted on

Dutch universities to take different approach to rankings

Dutch universities will take a more critical stance towards global university rankings. These ‘league tables’ have methodological shortcomings and their assessment of the research achievements of universities is too one-sided, being based on the number of publications and citations. This is at odds with the Recognition & Rewards programme, through which Dutch universities emphasise quality rather than quantity. So concludes an expert group advising the Board of the Universities of the Netherlands. UNL President Pieter Duisenberg: ‘Universities are ambivalent about university rankings. It’s important to be able to determine our place among the world’s top universities, but the current approach does not do justice to the breadth of work that we do. As a consequence, we’ll be using these rankings less frequently, contributing instead to the development of alternatives.’

What are rankings?

University rankings come in different shapes and sizes. The expert group’s advice focuses specifically on league tables. These are rankings that try to capture a university’s performance in an overall score of accomplishments in research, education and impact. In doing so, they claim to reflect the overall performance of a university. However, there is no universally accepted criterion to quantify a university’s overall performance. In addition, league tables emphasise research achievements, which are largely determined by the number of publications and citations.

Universities believe it is important to perform well in these international rankings, as many students, scientists, businesses and governments use them. However, the methods by which these league tables are drawn up and the value assigned to them is at odds with the principles of the Recognition & Rewards programme. As part of this programme, institutions actually want to give more weight to different qualities in science, rather than looking at a single quantitative dimension to measure quality. UNL therefore asked an expert group for advice on how universities can best deal with the league tables in relation to Recognition & Rewards.

 

What will change?

The expert group has indicated that the use of league tables can undermine Recognition & Rewards. It offered a number of recommendations at the institutional, national and international levels. In the coming period, Dutch universities will work together to implement these recommendations. Universities will make changes regarding the use of league tables at both the national and institutional levels. For example, universities will contribute to alternatives to league tables, such as multidimensional rankings. They will also no longer use the league tables for internal evaluations or budget allocations. In addition, the data supplied to league tables will be made public. At the national level, UNL will work to increase awareness about the limitations of league tables. It will also try to discourage the use of league tables by the government.

Furthermore, UNL will engage with international partners to bring about change at the European level as well. As a first step, UNL has taken the initiative to explore a joint guideline with European university umbrella organisation EUA and experts from Germany, France, the Republic of Ireland, Poland and the UK.

The expert group’s advice focuses specifically on league tables. There are many more rankings. These include topical rankings, subject rankings, regional rankings and rankings for universities that have existed for less than 50 years. The advice limits itself to league tables because they serve the largest group and unfairly claim to capture the performance of universities in one-dimensional rankings.

Source: https://www.universiteitenvannederland.nl/en_GB/nieuws-detail.html/nieuwsbericht/915-p-nederlandse-universiteiten-gaan-voortaan-anders-om-met-rankings-p

Posted on

New international agreements science funders on Recognising and Rewarding Researchers

Research funders from around the world have signed the Statement of Principles on Recognising and Rewarding Researchers. The statement sets out principles of how funders will work together to achieve new ways of assessing research and researchers, appropriate to the current times and in line with today’s researcher. ‘This is an important next milestone in an internationally supported process toward more balanced and fairer assessment of research,’ said Arfan Ikram, president of ZonMw and member of NWO Executive Board responsible for the Recognition & Rewards portfolio.

The statement states that there is growing momentum for reform toward broader and more inclusive assessment of research, but also that there are still challenges, which organizations face to make such changes. To help alleviate these challenges and achieve new ways and means of assessing research worldwide, research funders have agreed on nine principles.

Qualitative approach

For example, it was agreed that during the assessment process of research(ers) there should be sufficient space the for relevant context, such as the field of study or career stage. Aspects such as team science or citizen science should also be considered in the assessment process. The funders also agree that the assessment of research primarily requires a qualitative approach, supported by an open and responsible use of quantitative indicators.

The full statement can be found here

Ikram on the agreements made: ‘The Dutch research funders NWO and ZonMw wholeheartedly endorse this statement and have long been working according to the principles agreed today. The way academics are recognized and rewarded today must do justice to the different academic qualities that exist. That way there is more room for the quality and appreciation of the plurality of their tasks. Today’s statement signed by funders from America to Japan helps enormously in this regard.”

Peace Palace

The Statement of Principles was signed in The Hague during the annual meeting of the Global Research Council (GRC), the collaboration of national research councils and research funders around the world. Minister Dijkgraaf of OCW opened the event, which takes place at the Peace Palace . This is the first time that the annual gathering is taking place in the Netherlands. NWO is sharing hosting with Brazilian sister organization São Paulo Research Foundation (FAPESP) from May 29 to June 2. More than 160 participants are attending this year, including some 65 “heads of research councils”.

Founded in 2012, the GRC consists of the (presidents of) national research councils and research funders around the world. Through this collaboration, members share knowledge and best practices within international science policy and jointly set strategic agendas. Each year, research funders from around the world come together during the GRC Annual Meeting.

Source: https://www.nwo.nl/nieuws/nieuwe-internationale-afspraken-wetenschapsfinanciers-over-erkennen-waarderen

Posted on

First ScienceGuide survey results released

Dutch academics are moderately positive about the Recognition & Rewards programme. That is the outcome of the survey of 314 respondents launched by the online platform ScienceGuide in early April. The expected impact of the Recognition & Rewards programme on their own profession is rated positively by almost half of the respondents (46% positive, 21% neutral). The impact of the programme on assessing their personal performance is rated positively by 39 per cent of respondents (33% neutral). We are pleased to read that at the stage the programme is in, Dutch academics are moderately positive about the changes for their own field and own performance.

Change process

We realise that with such a major culture change, questions and uncertainties also arise. In the midst of this change process, it is therefore pleasing that the majority of respondents indicate a positive perception of the changes resulting from the Recognition & Rewards programme. Moreover, we see that respondents view the evaluation of various scientific activities such as education and open science positively. Especially compared to the current situation in which the evaluation of open science, side activities and administrative matters receive insufficient marks. The results of ScienceGuide’s research fit with the approach of the Recognition & Rewards programme which aims to recognise and reward all core tasks of a scientist.

Difference between disciplines

In the survey ScienceGuide also looked at the results of respondents from different subject areas. What is prominent here is that beta scientists are above average satisfied with the current situation. They expect the Recognition & Rewards programme to have a negative impact on the assessment of research. In local committees at universities, there is a lot of focus on involving scientists with different scientific disciplines. The results of this survey show even more that it is important to ensure that the implementation of the programme suits the work in practice. As a programme team, we therefore continue to seek dialogue and strive for national alignment in terms of assessment criteria. For instance, the road map Room for everyone’s talent in practice, published earlier this year, states that in 2024 we will clarify which quality characteristics we will use in the various core domains. We thereby take into account differences between disciplines.

Concerns among postdocs

When zooming in on the stage of the career the respondent is in, differences can be seen between professors and PhDs and between PhDs and postdocs. ScienceGuide indicates in its article that PhDs are pleased with the Recognition & Rewards programme, but postdocs have concerns about it. We recognise these concerns, which were also discussed at the recent Recognition & Rewards Festival in Utrecht. We are currently discussing which role postdocs and PhDs can take within the programme, so that they are even more actively involved in the implementation of the programme.

Recognition & Rewards ‘Cultuurbarometer’

ScienceGuide’s research provides interesting insights. We are always interested in data and information that offers the programme insights and can therefore help it move forward. We therefore look forward with interest to the next updates with results from ScienceGuide.  From the Recognition & Rewards programme we also have a survey planned for later this year. The Recognition & Rewards Cultuurbarometer is an initiative from the joint programme Recognition & Rewards and will be circulated among staff at participating institutions.

Posted on

Eerste resultaten enquête ScienceGuide bekend

Nederlandse academici zijn volgens ScienceGuide gematigd positief over het programma Erkennen & Waarderen. Dat is de uitkomst van de enquête onder 314 respondenten die het online platform ScienceGuide begin april lanceerde. De verwachte uitwerking van het Erkennen & Waarderen programma op het eigen vakgebied wordt door bijna de helft van de respondenten positief beoordeeld (46% positief, 21% neutraal). De invloed van het programma op de beoordeling van hun persoonlijke prestaties wordt door 39 procent van de respondenten positief beoordeeld (33% neutraal). We zijn verheugd te lezen dat in de fase waarin het programma zich bevindt Nederlandse academici positief zijn over de veranderingen voor het eigen vakgebied en de eigen prestaties.

(English translation >>)

Veranderproces

Vanuit het programma beseffen we dat met een dergelijke grote cultuurverandering ook vragen en onzekerheden naar voren komen. Midden in dit veranderproces is het daarom prettig dat de meerderheid van de respondenten aangeeft de veranderingen die voortvloeien uit het programma Erkennen & Waarderen positief te ervaren. Daarbij zien we dat respondenten de evaluatie van verschillende wetenschappelijke activiteiten zoals onderwijs en open science onder Erkennen & Waarderen positiever beoordelen. Zeker ten opzichte van de huidige situatie waarin de evaluatie van open science, nevenactiviteiten en administratieve zaken een onvoldoende krijgen. De resultaten van het onderzoek van ScienceGuide passen bij de insteek van het programma Erkennen & Waarderen waarin wordt beoogd om alle kerntaken van een academicus te erkennen en waarderen.

Verschil tussen disciplines

ScienceGuide heeft in haar onderzoek ook gekeken naar de resultaten van respondenten uit verschillende vakgebieden. Opvallend daarbij is dat bèta’s bovengemiddeld tevreden zijn met de huidige situatie. Zij verwachten dat het programma Erkennen & Waarderen negatieve gevolgen heeft voor de waardering van onderzoek. Bij de lokale commissies op de universiteiten is er veel aandacht voor het betrekken van wetenschappers met verschillende wetenschappelijke achtergronden. De resultaten van dit onderzoek tonen des te meer aan dat dit van groot belang is, om te waarborgen dat de concretisering van het programma blijft aansluiten bij de praktijk. Als programmateam blijven we daarom de dialoog zoeken en streven we naar landelijke afstemming wat betreft de beoordelingscriteria. Zo is onder andere opgenomen in de routekaart Ruimte voor ieders talent in de praktijk, die eerder dit jaar is gepubliceerd, dat we in 2024 duidelijk maken welke kwaliteitskenmerken we hanteren op de verschillende kerndomeinen. We houden daarbij rekening met verschillen tussen disciplines.

Zorgen bij postdocs

Als er ingezoomd wordt op het stadium van de carrière waarin de respondent zich bevindt zijn er verschillen te zien tussen hoogleraren en promovendi en ook tussen promovendi en postdocs. ScienceGuide geeft in haar artikel aan dat promovendi blij zijn met het programma Erkennen & Waarderen, maar dat postdocs er zorgen over hebben. Wij herkennen deze zorgen die ook werden besproken tijdens het recente Recognition & Rewards Festival in Utrecht. Daarom kijken wij welke rol postdocs en promovendi kunnen innemen binnen het programma, zodat zij nog meer betrokken zijn bij de implementatie van het programma.

Erkennen & Waarderen Cultuurbarometer

Het onderzoek van ScienceGuide levert interessante inzichten op. We zijn altijd geïnteresseerd naar data en informatie die het programma inzichten biedt en daardoor vooruit kan helpen. We kijken daarom met interesse uit naar de volgende updates met resultaten vanuit ScienceGuide. Vanuit het programma Erkennen & Waarderen staat ook een vragenlijst gepland voor later dit jaar. De Erkennen & Waarderen cultuurbarometer is een initiatief vanuit het gezamenlijke programma Erkennen & Waarderen en zal worden uitgezet onder medewerkers van de deelnemende instellingen.