Research funding is broken. Using a lottery approach could fix it

Thirty years ago, Katalin Karikó had what was then an outlandish idea: use messenger RNA as a medicine. But getting funding to demonstrate that might be possible was impossible, despite three decades of trying. One rejection she might now be able to laugh about was a research opportunity with funding for six scientists. It received seven applications — Karikó’s was the only one that wasn’t funded.

Her outlandish idea has since provided the platform for the vaccines that are helping protect people around the world from Covid-19 and is being used to develop other treatments. Karikó has since won multiple prizes and could even win a Nobel Prize.

Karikó overcame the lack of funding with incredible dedication. But her story illustrates that the world has likely missed out on other revolutionary scientific ideas because funding systems are designed to reward those who are already successful, not necessarily those who work represents real innovation.

Research funding is broken, a view widely shared among scientists. Typical funding rounds from national research agencies entail grueling processes: Scientists write lengthy applications which are then scrutinized by several assessors, often over multiple rounds of reviewing. After all this time and effort, the percentage of success is crushingly low, often just 10% or 20%.

This competitive model, which is almost universally used, has well-documented flaws. It is excessively wasteful in terms of researchers’ time. If only 20% of grants are funded, every dollar given by a research grant agency represents many valuable hours spent by skilled researchers writing appealing grant proposals, filling forms, and jumping through the hoops of the reviewing process. These are hours which were channeled away from fruitful work to advance science, create new medical therapies, inform policymakers, and generate innovations. Into this black hole of lost time must be added the hours university administrators spend polishing applications instead of contributing to other university duties.

Insights from contest theory show that this competitive model can lead to the total cost expended by applicants being equal to the resources provided. In Australia, one of the largest health and medical research competitions cost more than 600 years of scientists’ time. This is equivalent to funding 600 one-year fellowships. Given that the two largest government funding bodies in Australia award around 635 fellowships per year, the funding system is simultaneously giving and taking away.

Given the cost of the standard application and review processes, there should be some assurance they are allocating funds to the most worthy applicants and the most promising projects. Indeed, it is because these allocation decisions are so important that applicants are asked to work so hard and so long on presenting their ideas, and why many reviewers are used to judge the applications.

But the meritocratic nature of this process is questionable. Research shows that, barring a minority of outstanding projects, grant winners and losers are not decided by a precise and objective identification of worthy and unworthy projects. Instead, the luck of the draw — who reviews what proposal and the opinions they hold — generally determines these outcomes.

Having varied opinions among scientists is generally a good thing: it means they pursue different paths and spread their efforts across a range of ideas. But this variance also affects funding, and applications are funded — or not — depending on the views of the majority. An analysis of one Australian medical funding competition showed that 59% of the 620 applications funded might not have been funded if a different set of scientists had assessed the proposals.

This luck-of-the-draw approach is why a novel funding procedure is gaining support: adopting some randomization to allocate funds.

It works like this: All applications are reviewed by experts, but only to decide if they are fundable. Awards are then made at random from among the fundable group. This approach retains quality control, but also acknowledges the power of reviewers to accurately predict winners.

Randomization has several benefits. First, it reduces the lengthy application processes researchers must go through, and also reduces the time spent by reviewers, as their task is simpler: to decide whether an application is good enough to be entered in the funding lottery.

Second, it reduces the risk of snowballing effects where past grant success is interpreted by reviewers as a clear signal of quality, leading to unequal allocations of funds that tend to favor the same researchers.

Third, randomization reduces the role of politics in the reviewing process, which can prevent some interesting ideas from being funded because of reviewers’ strong views or competing interests.

Fourth, it would reduce constraints on the idea generation process. Knowing that a project does not have to be supported by all reviewers to possibly receive funding would allow researchers to propose more novel ideas. One likely benefit would be the greater opportunity to get funding for interdisciplinary research whose projects often die under the criticisms of gatekeepers who ask every application they review to follow the academic codes and views from their specific discipline.

Another benefit of randomized funding is that it creates a perfect randomized trial of proposals that win funding and those that didn’t, making it possible to follow up the work and measure the impact of funding. How much does research on climate change, virology, or economic policy, for example, affect societal outcomes? Randomized studies provide the strongest scientific evidence, as they are generally freer from biases than other types of research. It’s extremely ironic that there has never been a randomized study of how science is funded.

Randomization is being used by some research agencies around the world. The New Zealand Health Research Council has been a pioneer of lotteries, using them since 2013, and this approach has been generally well accepted by applicants. The Foundational Questions Institute in New York has used lotteries to fund research in cosmology and physics since at least 2013. In September, the British Academy, which focuses on humanities and the social sciences, announced that all applications deemed as “fundable” would be entered into a lottery. It follows recent announcements in Switzerland, Austria, and Germany.

To be sure, using research lotteries is not straightforward. The effort spent in the existing funding processes aims to ensure the decisions are right. It may be difficult to accept losing that goal. Yet by adopting randomization, funders and seekers of funding would only be dropping the illusion that existing decisions are right.

Katalin Karikó’s research projects were often rejected by the funding system but, luckily for the world, she is a remarkably persistent person. Paradoxically, by codifying luck in the funding system, less luck may be needed to create scientific breakthroughs.

Lionel Page is a professor of economics at the University of Queensland. Adrian Barnett is a professor of statistics at Queensland University of Technology.