That’s what one researcher told us when we asked them about applying for NHMRC Project Grant funding. Others said that applying for funding had made them ill, lost them friends, ruined Christmas and caused arguments with friends and family.
What makes applying for funding so bad? We’ve tried to summarise the problems with the system in the diagram above. This is based on our group’s four years of research into the funding process. Some of the arrows are based on evidence from our surveys (Survey 1, Survey 2), others are based on anecdote or experience and so maybe wrong. Please let me know if I’ve missed an arrow or an issue.
The limited funds are the most important problem, and every other problem can be traced back to this. The current low tide in funding has made the problems obvious, which is good for researching this area, but bad for Australian researchers. If more money flooded the system (as some people hope the government’s Medical Research Future Fund will) then some of these problems will disappear, but they will still be below the surface causing less obvious inefficiencies and problems, and will reappear during the next inevitable funding drought.
Limited funds cause low success rates and we know that researchers react to a lower chance of success by increasing their application numbers in order to get some funding. This creates a feedback loop that further reduces the success rate and so increases the number of applications. However, this association may be non-linear, as once success rates become very low then some researchers may give up applying or leave research completely. That’s certainly how some researchers felt on Twitter after the most recent funding announcement. If lots of researchers do give up then the next round could be easier for those prepared to try again. Unless lots of people work this out, in which case the next round might get even harder. (I’m reminded of my undergraduate lectures in game theory).
Both the NHMRC and ARC would like to receive fewer applications, and both believe that universities and research institutes should do more in-house culling. But is culling in an institution’s best interests? Let’s take QUT as an example. Last year they submitted 93 Project Grants and won 11, a success rate of 12%. Let’s imagine they had culled half the grants (47), how many winners would they have lost? Well given that we found that researchers submitting two grants could not predict which one would be funded, the safest assumption is that any culling process would also be insensitive and they would have lost half their winners. So pleasing the funding agency would have cost $2.6 million, which is not a good trade.
QUT could gain staff time by submitting fewer applications. Each Project Grant takes around 38 working days, so if the cull was made early enough that could be 38 × 47 = 4.8 years of research time or around $576K in staff costs. Perhaps QUT could pay researchers not to submit (in the same way that farmers are sometimes paid not to grow crops), but that simply returns us to the original problem of money.
Given the costs and over-competitiveness of the system, it’s no surprise that research from Canada found that simply dividing the funding pool equally amongst all applicants ($30K per applicant) was more efficient than bothering with preparing and peer reviewing applications (which cost $40K per application).
Gaming
A low success rate drives some researchers to gaming, or as some people might call it, cheating. Writing a retrospective application where the work has already been done, is a well-established technique that was pointed out by Osmond in 1983, “It has become almost universal practice for researchers to ‘propose’ what they have already done”.
We also know that a small percentage of researchers arrange conflicts of interest with other Australian researchers. If you think someone is likely to nobble your proposals then you invite them to be an author on a paper, which then stymies their nobbling. If you think someone is likely to be helpful then you never work with them and then suggest them as a reviewer. A consequence of this is lost productivity, as potentially interesting collaborations are foregone.
Saving time
By far the bigger driver of lost productivity is the time researchers spend writing applications. We estimated that 550 working years went into the 2012 Project Grant round. And after the exhaustion of writing applications the last thing researchers want to do is review their colleagues’ 50 page applications, preferring to get back to actual research.
A lot of time could be saved if there was a shorter initial expression of interest (EOI) followed by a longer application for those whose ideas were deemed interesting enough to hear more from. The argument against this is that it was tried once (in 2006) and applications numbers spiked. We know from our surveys that 33% of researchers said they would submit more applications if there was an EOI, and we also know that EOIs are popular with 73% supporting this change.
It is strange to argue against an EOI by worrying about the increase in applications, as this means that more ideas are being tested which has to be a good thing. EOIs could also help with the struggle to find reviewers. Full Project Grant applications are around 50 pages long. If the EOI was 2 pages then one reviewer could potentially read 24 more applications.
Appeals
Another unfortunate loop in the system is that failure fuels discontent which then fuels appeals. An example is the researcher who took the ARC to court, and won a concession. Funding bodies react to appeals by reducing feedback to reduce potential ammunition. But this means very poor feedback, sometimes just a sentence for two months’ work. Hence researchers have no idea how to improve their applications and quality declines. It’s also a terrible waste of resources considering that the best minds in the country have been gathered to evaluate your work, but you’ll never hear their thoughts.
AusHSI
We have always tried to do things differently at AusHSI. We have short applications of just 1,200 words, we give honest feedback and we work with applicants to improve their applications. Our applications take around 7 working days to prepare compared with 38 for the NHMRC. We ask for feedback from applicants about our funding system and we have experimented with the review process, including recording our discussions and giving a transcript to applicants. Giving better feedback to applicants has meant that we’ve seen dramatic improvements in proposals that we have then funded when they were re-submitted.
Obviously it’s easier to do this when your budget is in the hundreds of thousands and not the hundreds of millions. However, just because handing out funding is a complex problem that doesn’t mean it has to have a complex solution. The current funding system is drowning in its own complexity. Simplifying the system could have benefits for applicants and the funding agencies.
Adrian Barnett