Executable science: reproducing neuroscience analyses through collaborative open source tools and practices.

Modern science relies on computational tools. In this interdisciplinary convergence, researchers are increasingly expected to share reproducible analysis code. In this talk, we argue that open source software practices are invaluable for reproducible science. We share experience and lessons learned from a small-scale reproducibility project in an academic neuroscience lab. The goal of the project was two-fold: i) to reproduce a set of published results in computational neuroscience ii) to do so while adopting a set of software development practices and using a suite of open-source analysis and publishing tools. We f irst outline the components of our workflow (the Python scientific stack, collaborative version control, code formatting, automatic code documentation parsing, code packaging, publishing interactive reports via Myst Markdown). We then discuss the main takeaways from our reproducibility attempt. First, while we succeeded in reproducing aspects of original results, discrepancies remained. The code and data, despite being openly available, were lacking documentation in critical parts, forming a barrier to understanding and thus reproducibility. Second, software development practices and tools, when adopted, are tightly coupled with scientific work; we frequently found ourselves reconsidering core analytical choices due to specific stages of our development workflow (e.g. deploying code documentation). Third, we find that choosing the right workflow tools, balancing the trade off between sophistication and achievability, is an important skill. In conclusion, certain software development practices should be an integral part of scientific analysis and publication workflows to equip researchers with the ability to deliver on open science promises.