On Monday 8th April, over 70 people (including early career researchers, professors, librarians, policy makers and funders) met in Brisbane to discuss this important question: What can researchers do to improve the quality of health and medical research in Australia?
We opened with an inspiring talk from QUT’s Vice-Chancellor and President, Professor Margaret Sheil, who also provided funding for the meeting. There was an extremely wide range of topics, which included: the culture change needed in research, p-hacking, research contracts that allow for the suppression of unfavourable results, systematic fraud in cancer studies, and examples of training programs to improve research practice. Most of the slides are available here.
What we need to do to change research culture @BrianNosek #improveresearch pic.twitter.com/xDd4yfruMT
— Ginny Barbour (@GinnyBarbour) April 7, 2019
The meeting’s key question of improving research can be framed in the context of research waste, which occurs in every part of the research process. What can we do to stem the losses and increase the impact of research on people’s lives?
To get the most out of the day, we used a pre-meeting survey. It was a short survey, but with two difficult questions. The first difficult question was:
- Of the following 20 problems in research, please select which five (or fewer) you think are the most important to address.
(You can read the full survey results here.)
There were two clear “winners” for the problems:
- Inappropriate academic incentives (“publish or perish” reward systems focus on citations, especially in “top” journals, rather than research resulting in real changes to practice).
- Insufficient training of researchers and supervisors in research reproducibility and basic study design.
As Bill Clinton (should have) said, “it’s the incentives, stupid”. “Publish or perish” is the perverse incentive that drives researchers to cut corners, p-hack and spin the results of their experiments.
However, incentives can also be a cop-out for bad behaviour, as described in this excellent blog by Tal Yarkoni. It’s hypocritical to call ourselves rigorous scientists who exalt the power of the scientific process, but then hurl those ideals out the window when faced with competition for funding or journal articles.
The second difficult question was:
- Which of the following 21 potential policy changes or actions would you endorse to help improve research in Australia?
The top two ranked policies were:
- Better and continued training in research methods at all career stages, ideally using a national model that is available to all researchers
- That reproducibility training and practice becomes integrated into Masters and PhDs
So training featured prominently in the answers to both the problem and policy questions. This is a clear call for action to those responsible for training the next (and also the current) generation of researchers. One idea might be that we have four-year PhDs with the first year spent learning the basics of running experiments – and recognise these as “fully licensed” PhDs.
I am reminded of the common need for Australian university undergraduates to get remedial mathematics lessons before starting their mathematical sciences degree. Students are arriving at both the undergraduate and postgraduate level without the necessary foundations for their course.
It may be unfair to ask early career researchers to spend another year on research’s lowest wages. But if their careers will be 30 years or longer, then one year to improve those 30 years is justifiable.
Personally, I was happy to see this policy change rank highly:
- Lobbying universities to downplay league tables (like the Times Higher Education tables) and focus on real benefits to patients and the public.
We’ve mentioned the incentives of researchers, but the incentives of university managers need examining too. Having league table position as a KPI for senior managers is a terrible idea and fuels the “publish or perish” bonfire.
The last question of the day was, “what was going to come out of the meeting?” I know that some attendees have committed to reducing waste and improving quality in their own practice and others have made useful new connections. The ranking of policies and discussion during the day have been insightful to me, and I will try to take these forward. Feel free to ask me about positive changes in the future and suggest your thoughts on changes needed. Given the positive feeling about the meeting, we might reflect on what has improved by repeating the meeting in April 2020.
By Professor Adrian Barnett, Principal Research Fellow – Statistics @aidybarnett