Tag Archives: Open Access

Open Research Pilot project: the funder view

Case study, Funder, ,

This is the sixth and final piece in our series of blogs to mark the end of the Open Research Pilot Project – a two-year initiative through which four University of Cambridge research groups worked with University Research Support and the Wellcome Trust’s Open Research Team to assess what would be needed to make all their research outputs openly available. In this blog, David Carr, Programme Manager for Open Research at Wellcome, provides his perspectives on the project.

START OF THE PROJECT

As a global research foundation dedicated to improving health for everyone through enabling great ideas to thrive, the Wellcome Trust has been a long-standing and passionate champion of open access to research publications and research data sharing. We are committed to ensuring that the outputs of the research we support can be accessed and used by bright minds around the world to help accelerate research and its application to improve health.

When we were approached by the team at Cambridge in 2016 with the idea for the Open Research Pilot, we were in the process of establishing a new dedicated Open Research team at Wellcome to help spearhead our work to advance openness. We were in the process of developing a new policy on managing and sharing data, software and materials (which was subsequently published in July 2017). We had also just launched Wellcome Open Research with F1000 as a new platform to enable our funded researchers to rapidly publish any research finding they wished to make available using a fully open and transparent review process.

We were quick to accept their proposal and join forces. The Pilot offered a chance to explore the opportunities and challenges facing our funded researchers in adopting open approaches, and the resources and support they require to do this. It also offered the potential to explore whether the Wellcome Open Research platform could help researchers in making their outputs available.

 

PROJECT IN PROGRESS

Our main input during the Pilot was through periodic update meetings with the research groups and Cambridge team to review progress and discuss emerging themes and issues. I found these discussions enormously valuable.

There is simply no substitute to hearing about and discussing the practical challenges and barriers to open science first hand from our researchers. As the previous posts have described, these highlighted important issues around data size and complexity; incentives and recognition; skills and training; and the funding and sustainability of data resources. These conversations were very timely for us and have helped to inform the development of several aspects of our Open Research activity at Wellcome.

The Pilot also helped to build on our relationship with the research support team at Cambridge and provided valuable insights on the issues facing universities in supporting researchers to manage and share their research outputs. The expertise and dedication of the team to supporting the researchers with whom they work was hugely impressive, and they must take considerable credit for proactively initiating and taking forward this project. It is great to see that Cambridge has now formalized its support for open research at an institutional level through its recently-published position statement, and I hope it continues to adopt a leadership role in this space.

While I think the Pilot was hugely worthwhile, inevitably there were challenges and lessons to learn. Personnel changes in the team at Cambridge inevitably caused some minor disruptions, and it is fair to say that from Wellcome’s perspective other priorities sometimes meant our contribution and focus on the Pilot was sometimes less than it could have been. On reflection, there were probably opportunities to better align and link the Pilot with other activities at Wellcome that were missed.

We were a little disappointed that the Groups involved didn’t utilize our new publishing platform, although delighted that the first Wellcome Open Research data note resulting from the Pilot has now arrived (as highlighted in the first blog in this series). We hope that other Wellcome funded research groups at Cambridge will consider trying out Wellcome Open Research, and see if it adds value for them in rapidly and openly sharing their research findings.

 

LOOKING AHEAD

Wellcome is committed to supporting our researchers in adopting open research approaches in ways that maximise the value of research outputs and enrich the research enterprise.

Through our Open Research programme, we are taking forward a range of activities to support this goal, including:

  • providing the Research Enrichment – Open Research funding scheme to enable existing Wellcome grantholders to apply for additional funds to enhance the impact of their funded research through opening up their research outputs;
  • running the Open Research Fund – an annual competition to support cutting-edge, innovative approaches to open research around the world;
  • enhancing our support for researchers to manage and share data, including through an ongoing pilot with Springer-Nature to make its Research Data Support service available to our funded researchers;
  • taking a lead as a funder in incentivizing open research and working with others to accelerate implementation of the San Francisco Declaration on Research Assessment (DORA);
  • developing our guidance on developing and funding output management plans as part of grant applications, and introducing new approaches;
  • continuing to develop the Wellcome Open Research platform as a venue for Wellcome researchers to share their research findings.

In addition, of course, we are also focusing on accelerating the global transition towards full and immediate open access to research publications with our partners in cOAlition S.

We are committed to continuing to work with our funded researchers and institutions to advance open research. We got a lot out of our participation in this Pilot, and look forward to continuing to work closely with our colleagues at Cambridge and across the institutions we support.

 

Published 13 March 2019

Written by David Carr, Programme Manager for Open Research at The Wellcome Trust

Creative Commons License

Reflections on Open Research – a PI’s perspective

Uncategorized, ,

Originally published 22 June 2017 Written by Dr Marta Teperek

As part of the Open Research Pilot Project, Marta Teperek met with Dr David Savage and asked him several questions about his own views and motivations for Open Research. This led to a very inspiring conversation and great reflections on Open Research from the Principal Investigator’s perspective. The main points that came out of the discussion were:

  • Lack of reproducibility raises questions about scientific rigour, integrity and relevance of work in general
  • Being open is to work in a team and be collaborative
  • Open Research will benefit science as a whole, and not the careers of individuals
  • Peer review remains a critical aspect of the scientific process
  • Nowadays, global collaboration and information exchange is possible, making the data really robust
  • Funders should emphasise the importance of research integrity and scientific rigour

This conversation is reported below in the original interview format.

Motivations for doing Open Research

Marta: To start, could you tell me why you are keen on Open Research and why did you decide to get involved in the Open Research Pilot Project?

David: Sure, but before we start I wanted to stress that when I make comments about science, these are very general comments and they don’t apply to anyone in particular.

So my general feeling is that I am very concerned and disappointed about the lack of research reproducibility in science. Lack of reproducibility raises questions about scientific rigour, integrity and relevance of work in general. Therefore, I am really keen on exploring ways of addressing these failings of science and I want to make a contribution to solving these problems. Additionally, I am aware that I am not perfect either and I want to learn how I can improve my own practice.

Were there any particular experiences which made you realise the importance of Open Research?

This is just the general experience of reading and also reviewing far too many papers where I thought that the quality of underlying data was poor, or authors were exaggerating their claims without supporting evidence. There is too much hype around, and the general awareness about the number of papers published in high impact journals which cannot be reproduced makes the move to more transparent and open approaches necessary.

Do we need additional rewards for working openly?

How do you think Open Research could benefit academic careers?

I am not sure if Open Research could or should benefit academic careers – this should not be the goal of Open Research. The goal is to improve the quality of science and therefore the benefit of science to the public. Open Research will benefit science as a whole, and not the careers of individuals. Science has become very egotistical and badge –accumulating. We should be investigating things which we find interesting. We should not be motivated by the prize. We should be motivated by the questions.

In science we have far too many people who behave like bankers. Publishing seems to be the currency for them and thus they are sloppy and lack the necessary rigour just because they want to publish as fast as they can.

In my opinion it is the responsibility of every researcher to the profession to try to produce data which is robust. It is fine to make honest mistakes. But it is not acceptable to be sloppy or fraudulent, or not to read enough literature. These are simply not good enough excuses. I’m not claiming to be perfect. But I want to constantly improve myself and my research practice.

Barriers to greater openness in research

What obstacles may be preventing researchers from making their research openly available?

The obvious one is competition for funding, which creates the need to publish in high impact factor journals and consequently leads to the fear of being scooped. And that’s a difficult one to work around. That’s the reason why I do not make everything we do in my research group openly available. However, looking at this from society’s perspective, everything should be made openly available, and as soon as possible for the sake of greater benefit to mankind. So balance needs to be found.

Do you think that some researchers might want to make their research open, but might not know how to do it, or might not have the appropriate skills to do it?

Definitely. Researchers need to know about the best ways of making their research open. I am currently trying to work out how to make my own project’s website more open and accessible to others and what are the best ways of achieving this. So yes, awareness of tools and awareness of resources available is necessary, as well as training about working reproducibly and openly. In my opinion, Cambridge has a responsibility to be transparent and open about its processes.

Role of peer-review in improving the quality of research

What frustrates you most about the current scholarly communication systems?

Some people get frustrated with the business model of some of the major publishers. I do not have a problem with it, although I do support the idea of pre-print services, such as bioRxiv. Some researchers get frustrated about long peer-review process. I am used to the fact that peer-review is long, and I accept it because I do not want fraudulent papers to be published. However, flawed peer review, such as biased peer-review or lack of rigorous peer review, is not acceptable and it is a problem.

So how to improve the peer-review process?

I think that peer-reviewers need to have greater awareness of the need for greater rigour. I was recently asked to peer review an article. The journal had dedicated guidance for peer reviewers. However, the guidance did not contain any information about suitability to undertake the peer-reviewing work. Peer-reviewer guidance documents need to address questions like: Do you really know what the paper is about? Do you know the discipline well enough? Are there any conflicts of interest? Would you have the time to properly peer-review the work? Peer-review needs to be done properly.

What do you think about the idea of journals employing professional peer-reviewers, who could be experts in their respective fields and could perform unbiased, high quality peer-review?

This sounds very reasonable, as long as professional peer-reviewers stay up to date with science. Though this would of course cost money!

I suppose publishers have enough money to pay for this. Have you heard of open peer-review and what do you think about it?

I think it is fine, but it might be subject to cronyism. I suspect that most people will be more likely to agree for their reviews to be made open as long as they make a recommendation for the paper to be accepted.

I recently reviewed a paper of a senior person and I rejected it. But if I made my review open, it would pose a risk to me – what if the author of the paper I rejected was the reviewer of my future grant application? Would they still assess my grant application objectively? What if people start reviewing each other’s papers and start treating peer-review as a mechanism to exchange favours?

The future of Open Research is in your hands

Who or what inspires you and makes you optimistic about the future of Open Research?

In Cambridge and at the Wellcome Trust there are many researchers who care about the quality of science. These researchers inspire me. These are very clever people, who work hard and make important discoveries.

I am also inspired by teamwork and collaboration. In Big Data and in human genetics in particular, people are working collectively. Human genetics and epidemiology are excellent examples of disciplines where 10-20 years ago studies were too small to allow researchers to make significant and reproducible conclusions. Nowadays, global collaboration and information exchange is possible, making the data really robust. As a result, human genetics is delivering really important observations.

To me, part of being open is to work in a team and be collaborative.

If you had a magic wand and if you could get one thing changed to get more people share and open up their research, what would it be?

Not sure… I suppose I am still looking for it! Maybe I will find one during the Open Research Pilot Project. Seriously speaking, I do not believe that a single thing could make a difference. It is the little things that matter. For example, on my side I am trying to make my own lab and institute more aware of reproducibility issues and ensure that I can make a difference in my own environment.

So as a Group Leader, how do you ensure that researchers in your own group are rigorous in their approach?

First, I really make them aware of the importance of reproducible research and of scientific rigour. I am also making a lot of effort to ensure that my colleagues are up to date with literature. I ask them if they read important literature and if they are unable to answer I ask them to do their homework. I am also imposing rigorous standards for experiments. In my lab people repeat the key experiments, or those which are particularly surprising, in a blind fashion. It takes a lot of time and extra resources, but it is important not to be too quick and to validate findings before making claims.

I am also ensuring that my people are motivated. For example, even though everyone helps each other in my group, all PhD students have direct access to me and we have regular discussions about their work. It is important that your group is of a manageable size; otherwise, as a group leader, you will not know all your people and you will not be able to have regular discussions about their work.

How do you identify people who care about reproducible research when making hiring decisions?

I ask all prospective applicants to make a short presentation about their previous work. During their presentation I ask them to tell me exactly what their research question was and how confident they were about their discovery. I am looking for evidence of rigorous methodology, but also for honesty and for people who are not overselling their findings.

In addition, I ask about their career goals. If they tell me that their career goal is to publish in Nature, or have two papers in Science, I count this against them. Instead, I favour applicants who are question-driven, who want to make progress in understanding how things work.

Role of funding bodies in promoting Open Research

Do you think that funders could play a role in promoting Open Research?

Funders could definitely contribute to this. The Wellcome Trust is a particularly notable example of a funding body keen on Open Research. The Trust is currently looking into the best ways to make Open Research the norm. Through various projects such as the Open Research Pilot, the Trust helps researchers like myself to learn best practice on reproducible research,and also to understand the benefits of sharing expertise to improve skills across the research community.

Do you think funder policies to mandate more openness could help?

Potentially. However, policies on Open Access to publications are easy to mandate and relatively easy to interpret and implement. It is much more difficult for Open Research. What does Open Research mean exactly? The right scope and definitions would be key. What should be made open? How? The Wellcome Trust is already doing a lot of work on making important research results available, and human genomic data in particular. But making your proteomic and genomic data publicly available is slightly different from ensuring that your experiments are rigorous and your results honest. So in my opinion, funders should emphasise the importance of research integrity and scientific rigour.

To close our discussion, what do you hope to achieve through your participation in the Open Research Pilot Project?

I want to improve my own lab’s transparency. I want to make sure that we are rigorous and that our research is reproducible. So I want to learn. At the same time I wish to contribute to increased research integrity in science overall.

Acknowledgements

Marta Teperek would like to thank SPARC EUROPE and Dr Joyce Heckman for interviewing her for the Open Data Champions programme – many of the questions asked by Marta in the interview with Dr David Savage originate from inspiring, open questions prepared by SPARC EUROPE.

Originally published 22 June 2017
Written by Dr Marta Teperek

Creative Commons License

We are going OPEN – the Open Research experiment has begun!

Uncategorized, ,

Originally published 08 February 2017
Written by Dr Marta Teperek

There has been much discussion recently about the reproducibility crisis and about the growing distrust among the public in the quality of research. As illustrated in our ‘Case for Open Research’ series of blog posts, one of the main reasons for this is that researchers are currently rewarded for the number of papers they publish in high impact factor journals, and not necessarily for the quality of work that they are doing.

Indeed, Cambridge researchers clearly indicated that the lack of incentives to do anything other than publishing in these types of journals is one of the main blockers discouraging them from adopting a more open research practice.

Joining forces with the Wellcome Trust

The Office of Scholarly Communication started talking about these problem with theOpen Research team at the Wellcome Trust. The Wellcome Trust are natural allies, as they have consistently led their researchers towards greater openness. They were one of the first funding bodies to introduce policies on Open Access and on data management and sharing. Now the Wellcome Trust is moving towards proactively supporting Open Research beyond enforcing their compliance requirements.

To promote immediate and transparent research sharing, they have recently launched the Wellcome Open Research platform which allows researchers to submit articles about virtually any research output and get published within a couple of days. The Wellcome Trust is now considering making Open Research one of their strategic priorities.

We quickly realised that we have a lot of shared interests, and joining forces to tackle the problem together made a lot of sense. We came up with the idea to launch the Open Research Pilot Project.

The Open Research Pilot – understanding the barriers to “openness”

We conceived the project as a two year experiment, which would allow us to gain an understanding of what is needed for researchers to share and get credit for all outputs of the research process. These include non-positive results, protocols, source code, presentations and other research outputs beyond the remit of traditional publications.

The Project aims to understand the barriers preventing researchers from sharing (including resource and time implications), as well as what the incentives are. The Project aims to utilise the new Wellcome Open Research publishing platform, together with other channels, to share these outputs.

The invitation to take part in the Pilot was sent to all researchers at Cambridge funded by the Wellcome Trust. Participating researchers had to commit to sharing of research outputs beyond traditional publications and to engage with the Project, by participating in Project meetings and contributing to Project publications.

Is ‘doing the right thing’ enough incentive?

Our biggest question was whether anyone would be willing to participate in the Pilot. We did not offer any incentive other than encouraging researchers to contribute to the greater good. The only support available to those who wanted to take part in the project was that offered by the Wellcome Trust and Cambridge Open Research team members, but there was no financial aid available to prospective participants. We thought that regardless of the outcome, that inviting researchers would be a good exercise to go through – we thought that if no one applied, we would have learnt that doing ‘the right thing’ was not a good enough motivator.

Thankfully, we received several fantastic applications from individual researchers and research groups who demonstrated great interest in and motivation for Open Research. We initially planned to work with two research groups, but given the quality of applications received and passion for Open Research expressed by the applicants, we decided to extend the scope of the project to four research groups. We have selected researchers doing different types of research, with the aim of learning about distinct problems in sharing that are experienced in diverse research disciplines:

  •       Dr Laurent Gatto –is  doing computational biology research, with a special focus on proteomics data. His interest is: How to effectively share research data and the code needed to reproduce them?
  •       Dr David Savage – is researching molecular pathogenesis of the consequences of obesity. His question is: What are the problems with sharing data coming from human participants?
  •       Dr Benjamin Steventon – is a developmental biologist generating and analysing large-scale imaging datasets. He would like to know: Are there image repositories allowing one to share large image datasets in a re-usable way?
  •       Dr Marta Costa and Dr Greg Jefferis (and others) – researchers leading the work on two collaborative projects: Connectomics and Virtual Fly Brain, which will create interactive tools to interrogate Drosophila neural network connections. They would like to understand: What are the issues with sharing complex interactive datasets? How to ensure long-term preservation of complex digital objects?

Motivations

So what motivated these researchers to apply for the project? We asked this question at the application stage and were positively surprised by the altruistic answers that we received. Our researchers were largely driven by a desire to improve the research process. We have seen responses like:

  • “Openness in research, from data and software to publication, is a central pillar of good research.”
  • “I am very concerned (disappointed as a scientist) by the current wave of ‘unreproducible’ and/or ‘irrelevant’ research, and am very passionate about contributing to improving scientific endeavour in this regard.”
  • “I am very enthusiastic about exploiting new ways of sharing my research output beyond the established peer-review journal system.”
  • “I believe that sharing research outputs fully, including data and code are essential to accelerate research, and I have benefitted from it in my own research.”

Summarising, researchers expressed a great desire for contributing to a cultural change. Researchers wanted to change the way in which research was disseminated and to increase research transparency and reproducibility.

Let’s get to work

We all met (the researchers, Wellcome Trust and Cambridge Open Research teams) on Friday 27 January to officially start the two year project. Each research group was appointed a facilitator – a dedicated member of the Cambridge Open Research team to support researchers during the Project. Research groups will meet with their facilitators on a monthly basis in order to discuss shareable research outputs and to decide on best ways to disseminate these outputs. Every six months all project members will meet together to discuss the barriers to sharing discovered and to assess the progress of the Project.

One of the main goals of the Project is to learn what the barriers and incentives are for Open Research and to share these findings with others interested in the subject to inform policy development. Therefore, we will be regularly publishing blog posts on the Unlocking Research blog and on the Wellcome Open Research blog with case studies describing what we have discovered while working together. There will be an update from each research group every six months. We will also be publicly sharing all main outputs of the Project.

We are all extremely excited about going “Open” and we suggest that anyone interested in the Open Research practice watches this space.

Originally published 08 February 2017
Written by Dr Marta Teperek
Creative Commons License