Towards an alliance-wide adoption of open science
In the first CIVICA Research open science roundtable, researchers and staff from the alliance discussed how to promote and deepen open science practices.
The online event, “Open Science, the Social Sciences and the CIVICA Alliance,” was co-organized by LSE and Sciences Po on 27 October 2021.
Susana Mourato, Pro-Director of Research and Professor in Environmental Economics at LSE, opened the event as lead of the Open Science work package for CIVICA Research. Mourato explained that it was really the Covid pandemic that accelerated change and proved that open science is the way of doing science in the 21st century: science that is open, collaborative, shared, transparent and responsible.
Mourato traced different moments that shaped her exposure to open science. In 2018, for example, Professor Paul Ferraro, a leading environmental economist from John Hopkins University, delivered a fascinating keynote on whether there was a replicability crisis in environmental and resource economics. He talked about widespread practices such as selectively reporting results or using designs with low statistical power, and he criticized the professional incentives that encourage these practices. For Mourato, this was a fascinating and thought-provoking talk that triggered change.
She now believes that open science will become the default model for research in a global world, an irreversible shift enabled by technology. The CIVICA alliance offers such opportunities to collaborate and raise awareness of the advancement of open science within member institutions, through developing common frameworks to support the communities of practice and research. It is particularly important to define what open science means in the context of social science research, as opposed to, for example, in STEM. Mourato stressed that the roundtable intends to start a conversation about an open science plan for the CIVICA alliance, with a focus on social sciences, and also more broadly to promote and foster best practices in open science across the eight member institutions.
Drivers and barriers in open science practices and in the social sciences
In the first panel, Patrick Dunleavy, Emeritus Professor of Political Science and Public Policy at LSE and Editor in chief of LSE Press, discussed with Denisa Kostovicova, Associate Professor in Global Politics at LSE, Miklos Koren, Professor of Economics at CEU and Matteo Galizzi, Associate Professor of Behavioral Science at LSE.
Dunleavy provided another perspective on open science using an illustration from The Turing Way handbook to reproducible, ethical and collaborative data science. He stressed the non-reproducibility or non-replicability of some data within the field of social sciences.
Koren took the floor to share his experience as an economist. As the data editor of the Review of Economic Studies, he explained that he revises articles and the supporting replication packages, to make sure that the standards for reproducibility are met. Koren considers this a difficult task, because the standards differ from one discipline to another. So, most of his job involves disciplines in which researchers are more likely to use secondary data, like administrative data or published statistics, and the data is still often confidential and proprietary.
Another observation Koren made is that journals in economics are increasingly not-for-profit (such as society journals) and this has implications for the capacity of their teams to educate and enforce open science standards. Additionally, authors are sometimes circumspect about the effort required to provide evidence in their mathematical demonstrations, compared to the expected benefits.
In contrast to this, Koren recalled in his conclusion a recent experience of writing an article about social distancing measures during the pandemic. He had to do things fast, and had considered reproducibility already at the start of the project. So, as he collected data, he saved the documentation and made sure that the licenses were all in order and that the data could be properly cited. Thanks to this approach, the turnaround for revision was very quick.
Matteo Galizzi provided further insights into open science practices. As a researcher in psychology with training in experimental and behavioral economics, he has noticed a crisis in reproducibility and a lack of incentives at both individual and institutional level to adopt open and transparent research practices. Galizzi stressed the need to highlight actions undertaken at the level of an alliance such as CIVICA or by networks, such as the UK Reproducibility Network (UKRN), which is a national peer-led consortium organized in 57 local networks.
This trend in social science reflects a more general trend in science. In 2005, a very provocative paper by John Ioannidis argued that most scientific results are actually false, including specifically medical randomized controlled trials. Galizzi explained that, in the social sciences, the tension between reproducibility and replicability comes from experimental psychology, which has spread rapidly to many social sciences disciplines. When research is published or made available, it is thus important to share the protocols, material, and the data. But this will not be enough for new scientific findings in experimental behavioral sciences, as we also want to move to general robustness. So, Galizzi argued, we have to study whether the results can be generalizable. A second consideration is to use powerful tools like the registration of the study protocol, which can then be made available to anyone who reviews it. This is how an alignment occurs between reproducibility, science transparency and openness.
The final panelist in the round table, Denisa Kostovicova, first discussed the question of visibility, arguing that it is related not only to the researcher but also to the topics and knowledge. Kostovicova gave the example of an article on gender-based violence from a political economy perspective - a marginal and unexplored topic, but which can get a huge number of views within days of publication, and this is very important in terms of democracy and democratization of knowledge. This is, crucially, also an issue of geographical equality to access articles, as it addresses both the question of who creates knowledge and the question of scrutiny by the researchers themselves but also by the research participants.
Kostovicova explained that LSE’s focus on the policy impact of research is not just about broadening theoretical horizons but also about how new knowledge impacts policy. Open access too is changing the game. When it comes to data, and especially qualitative data, the issues and tensions within commitments to research ethics are rising. Anonymization does not necessarily solve the problem, but sharing the data and being transparent are extremely important for the legitimacy of the research, especially when the arguments of the researchers can be easily dismissed based on ideology.
Regarding quantitative data, researchers may have been locked in a false dichotomy between quantitative and qualitative research, especially from the perspective of research ethics dilemmas that come into tension with other drivers for publishing data.
Another issue is the sequencing of the data publication: at which point can the data be published and how does that fit in with the project cycle, and with the need of publishers when an article is submitted for example?
The development of collaborative open science within the CIVICA alliance
The second roundtable was opened by Guillaume Plantin, Professor of Economics, Vice President for Research at Sciences Po and scientific coordinator of CIVICA Research. The whole CIVICA project, Plantin explained, favours a bottom-up approach, placing researchers and faculty first. For CIVICA Research, a dedicated work package with an adequate budget was set up for open science. The aim is to learn by doing and understand what is most useful for the researchers.
Plantin also argued that CIVICA is a highly specialized European University and perhaps the most homogeneous alliance, in the sense that all members are active internationally in their own country, for example open to global economic activity. There are strong specificities among the social sciences regarding open science, so it is important to have a lobby approach within CIVICA to make sure that the alliance has a powerful voice in the open science debate. Additionally, open science is particularly relevant for PhD researchers, as a new and innovative way to do research, and that is why CIVICA Research includes a programme of PhD ambassadors on open science with a training approach. In his conclusion, Plantin referred to the weakening of democracy at the beginning of the 21st century, arguing that open science can be a way to fight the rising skepticism towards scientific expertise and social science expertise.
At the end of the event, Denisa Kostovicova called for some self-reflection on what it means to be an open scientist with regard to legitimacy and expertise. Researchers must focus on the practicalities of open science, and it is important to understand both the advantages of broader contributions to science and the constraints. Training PhD students is important, and this can be broadened to a larger community. Learning from each other in a collaborative way could be highly beneficial for everyone involved in open science within the alliance. Kostovicova also noted it is important to address issues of inequalities, for example through external collaborations in areas which are typically disadvantaged in the academic world.
Watch the video recording of the roundtable>>
CIVICA Research brings together researchers from eight leading European universities in the social sciences to contribute knowledge and solutions to the world’s most pressing challenges. The project aims to strengthen the research & innovation pillar of the European University alliance CIVICA. CIVICA Research is co-funded by the EU’s Horizon 2020 research and innovation programme. To stay up to date with CIVICA Research developments and opportunities, subscribe to the newsletter.