2012


Dec 10-14, 2012-- ICERM Workshop "Reproducibility in Computational and Experimental Mathematics," Brown University.

In addition to advancing research and discovery in pure and applied mathematics, computation is pervasive across the sciences and now computational research results are more crucial than ever for public policy, risk management, and national security. Reproducibility of carefully documented experiments is a cornerstone of the scientific method, and yet is often lacking in computational mathematics, science, and engineering. Setting and achieving appropriate standards for reproducibility in computation poses a number of interesting technological and social challenges. The purpose of this workshop is to discuss aspects of reproducibility most relevant to the mathematical sciences among researchers from pure and applied mathematics from academics and other settings, together with interested parties from funding agencies, national laboratories, professional societies, and publishers. This will be a working workshop, with relatively few talks and dedicated time for breakout group discussions on the current state of the art and the tools, policies, and infrastructure that are needed to improve the situation. The groups will be charged with developing guides to current best practices and/or white papers on desirable advances.


2011


July 16, 2011-- Community Forum on Reproducible Research Policies, Vancouver, British Columbia.

Reproducibility has been a cornerstone of theoretical and experimental science for hundreds of years, but the increasing reliance on and sophistication of computational methods too often makes replication of results difficult or impossible today. In this minisymposium we examine best-practices and proposals for encouraging reproducibility in computationally driven research at both a personal and community level through talks highlighting the outcomes of a pre-ICIAM satellite workshop on the same theme.


July 13-15, 2011-- Reproducible Research: Tools and Strategies for Scientific Computing. Applied Mathematics Perspectives 2011 at the ICIAM Satellite Meetings, Vancouver, British Columbia.

See the workshop website (including talk abstracts, slides, and video) here: http://stodden.net/AMP2011/.

Computation has become a vital component of research in the applied areas of mathematics, and through them all areas of science and engineering. Academic publications or industrially relevant mathematical results that do not involve some aspect of computational analysis are few and far between. Unfortunately, the software and data that drives this computation is too often developed and managed in a haphazard fashion prone to error and difficult to replicate or build upon.


Feb 19, 2011-- The Digitization of Science: Reproducibility and Interdisciplinary Knowledge Transfer, a symposium at the AAAS Annual Meeting, Washington, DC.

Scientific computation is emerging as absolutely central to the scientific method, but the prevalence of very relaxed practices is leading to a credibility crisis affecting many scientific fields. It is impossible to verify most of the results that computational scientists present at conferences and in papers today. Reproducible computational research, in which all details of computations -- code and data -- are made conveniently available to others, is a necessary response to this crisis. This session addresses reproducible research from three critical vantage points: the consequences of reliance on unverified code and results as a basis for clinical drug trials; groundbreaking new software tools for facilitating reproducible research and pioneered in a bioinformatics setting; and new survey results elucidating barriers scientists face in the practice of open science as well as proposed policy solutions designed to encourage open data and code sharing. A rapid transition is now under way -- visible particularly over the past two decades -- that will finish with computation as absolutely central to scientific enterprise, cutting across disciplinary boundaries and international borders and offering a new opportunity to share knowledge widely.


2009


Nov 21, 2009-- Data and Code Sharing Roundtable, Yale Law School.

A small group of leading computational scholars and stakeholders from funding agencies and journals gathered to discuss reproducibility in computational science.
Further details, including attendees, the agenda, and contributed thought pieces are available on the roundtable website.