Body in Mind and measuring impact
Measuring the impact of publishing research via a blog has been a reoccurring theme here. As part of my work with BodyInMind.org – a clinical research site which publishes on the topic of pain research – we are looking at whether dissemination makes any difference to the impact of published research.
Recently Campus Review interviewed Lorimer Moseley (the lead clinical scientist of BiM) about this topic. It is an excellent article and worth reading the full report here: The blog rankings. A different way of measuring impact.
Lorimer is Professor of Clinical Neurosciences and University of South Australia’s inaugural Chair in Physiotherapy where he works as a clinician and researcher with a special interest in pain and brain sciences. We set up BodyInMind to look particularly at the role of the brain and mind in chronic and complex pain disorders. This website includes links to published articles, current projects, teaching resources for clinicians and lecturers, books, seminars and conferences and other info that the team thinks is ‘intriguing, important or irresistible’.
“As a clinical scientist, what I am trying to do is change clinical practice, and one of the ways I am evaluated as a scientist is by my impact – and if I am trying to impact people who never write papers, my citation count is not going to reflect what we are doing, and in any case, writing papers will take ten years to get to the punters who use it on the coalface,” Moseley said.
So two and a half years ago, working with Heidi Allen, a digital publishing and social media specialist with an interest in public health, Moseley launched BodyinMind. “We would try to tap into social media to disseminate information that was understandable, that got people interested and enthusiastic about research,” he said. “There is a huge gap that exists between researchers and clinicians… they are distrustful I think, and we were wanting to break that down.”
He is aided by a team of researchers, including postgraduate students, and the site now attracts patients along with clinicians, policy makers and industry representatives. But the bigger question is, how can the site’s impact be measured? “I think it remains a bit of an experiment, certainly within the [National Health and Medical Research Council] system,” said Moseley.
“The idea that one’s impact might be measured in slightly less conventional ways is not an idea that I’ve come across before. I guess when I go for my next fellowship I will be taking the punt in including BodyinMind as a measure of my impact.” Part of the problem is that measuring social media “reach” is still a very new science. “It’s very difficult to work out what impact one has – we’ve got the conventional measures of followers and hit rates, but that doesn’t measure impact,” he said. Moseley has his team working a few experiments to measure affects on the blog on the dissemination of information, including randomised studies involving open access articles and their download rates.
“I don’t know of anyone who has really tried to engage with measuring it, and I see dissemination as a critical role of the scientist, and I don’t think it is done very well.” In some ways, this uncertainty has made the entire blog something of a risk. “I think there is a serious chance I’ll get a panel [at the NHMRC] that will say, ‘that’s not really a measure’ [of impact],” said Moseley.
“We’ve scoured the literature and no one can really prove what seems to be accepted, that social media is the best way of disseminating thoughts. Everyone quotes this nebulous idea that you have to have a social media evidence to get things done…. It’s intuitively sensible that it should be important but does it do anything? I don’t think anyone can claim to have evidence.”