Quis Judicabit Ipsos Judices? A Case Study on the Dynamics of Competitive Funding Panel Evaluations

Autor/in	Santos, João M.
Titel	Quis Judicabit Ipsos Judices? A Case Study on the Dynamics of Competitive Funding Panel Evaluations
Quelle	In: Research Evaluation, 32 (2023) 1, S.70-85 (16 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Santos, João M.)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0958-2029
DOI	10.1093/reseval/rvac021
Schlagwörter	Foreign Countries; Research; Financial Support; Peer Evaluation; Researchers; Social Science Research; State Aid; Qualitative Research; Evaluation Criteria; Decision Making; Portugal + Suchen Sie Ihr Suchwort? Ausland; Forschung; Finanzielle Förderung; Researcher; Forscher; Social scientific research; Sozialwissenschaftliche Forschung; Qualitative Forschung; Decision-making; Entscheidungsfindung
Abstract	Securing research funding is essential for all researchers. The standard evaluation method for competitive grants is through evaluation by a panel of experts. However, the literature notes that peer review has inherent flaws and is subject to biases, which can arise from differing interpretations of the criteria, the impossibility for a group of reviewers to be experts in all possible topics within their field, and the role of affect. As such, understanding the dynamics at play during panel evaluations is crucial to allow researchers a better chance at securing funding, and also for the reviewers themselves to be aware of the cognitive mechanisms underlying their decision-making. In this study, we conduct a case study based on application and evaluation data for two social sciences panels in a competitive state-funded call in Portugal. Using a mixed-methods approach, we find that qualitative evaluations largely resonate with the evaluation criteria, and the candidate's scientific output is partially aligned with the qualitative evaluations, but scientometric indicators alone do not significantly influence the candidate's evaluation. However, the polarity of the qualitative evaluation has a positive influence on the candidate's evaluation. This paradox is discussed as possibly resulting from the occurrence of a halo effect in the panel's judgment of the candidates. By providing a multi-methods approach, this study aims to provide insights that can be useful for all stakeholders involved in competitive funding evaluations. (As Provided).
Anmerkungen	Oxford University Press. Great Clarendon Street, Oxford, OX2 6DP, UK. Tel: +44-1865-353907; Fax: +44-1865-353485; e-mail: jnls.cust.serv@oxfordjournals.org; Web site: http://applij.oxfordjournals.org/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01