|
|
Joint Quantitative Brownbag (Joint QBB) Speaker Series, Sept 9, 2024.
|
AI-enabled Discovery: The Digital Scholarly Record.
| |
2024 US-UK Scientific Forum
on science in the age of AI, June 12, 2024.
|
On Emergent Limits to Knowledge
Or, How to Trust the Robot Researchers: A Pocket Guide.
|
Keynote,
Learning from Authoritative Security Experiment Results (LASER) Workshop, December 5, 2023.
|
A Decade Later: Reproducibility & Reliability of
Research Results.
|
Departmental Seminar, Alfred-Weber-Institut Für Wirtschaftswissenschaften,
Heidelberg University, Germany, November 30, 2023.
|
Automating Assessment of Machine Learning Research:
Revisiting Arrow’s Impossibility Theorem.
|
Digital Neurosurgery 2023, Regulating Digitization: Legal, IP, Privacy and Ethical Issues, October 14, 2023.
| Is Neurosurgery a Creative Work? (and why do we care?).
|
Joint Statistical Meetings, Statistics and the Reproducibility Crisis, August 10, 2023.
|
Automating Assessment of Computational Reproducibility in Machine Learning Research.
|
REAL@USC-META CENTER First Anniversary Conference, October 31, 2022.
| Automating Machine Learning Model Checking.
|
Building a Community Roadmap to Robust Science
in High-Throughput Applications, SIAM Conference on Computational Science and Engineering (CSE21), March 2, 2021.
| Advancing Computational Scientific Discovery by Enabling Reproducibility and
Transparency: Policies and Practice
|
Scholarship for a Post-Pandemic World: A Conversation; Council of
Graduate Schools 60th Annual Meeting, December 3, 2020.
| Training and Scholarly Impact in a Digital World.
|
Epstein Institute Seminar, USC, September 1, 2020.
| Two Projects for Advancing Scientific Reliability in Complex Computational and Human Systems.
|
Keynote, Applied Human Factors and Ergonomics Conference: The Human-Side of Service Engineering, July 18, 2020.
| Toward a Computable Scholarly Record: Meta Science and Engineering Reproducibility in the Era of AI.
|
P-RECS'20 3rd International Workshop on Practical Reproducible Evaluation of Systems, June 24, 2020.
| Beyond Open Data: A Model for Linking Digital Artifacts to Enable Reproducibility of Scientific Claims.
|
P-RECS'20 3rd International Workshop on Practical Reproducible Evaluation of Systems: Open Discussion, June 24, 2020.
| ACM Reproducibility and Replication EIG.
|
CS IEEE Ad Hoc Committee on Reproducibility and Open Science, June 17, 2020.
| "Reproducibility and Replication in
Science” Report Recommendations.
|
Realizing Opportunities for Advanced and Automated Workflows in Scientific Research: Second Meeting, March 16, 2020.
| Cyberinfrastructure Shapes Scientific Outcomes in Crucial and
Largely Unrecognized Ways.
|
Lightning Talk, Vice Chancellor for Research and Innovation visit to the School of Information Sciences, University of Illinois at Urbana-Champaign, Feb 21, 2020.
| Reproducibility for Computational and Data-Enabled Science.
|
AAAS Annual Meeting, Panel: The Reproducibility Revolution:
Impacts on Science, Journalism, and Society, Feb 15, 2020.
| Reproducibility Across Research Disciplines and Stakeholder Communities.
|
SIAM Conference on Parallel Processing for Scientific Computing (PP20), Feb 13, 2020.
| Transparency and Reproducibility:
Case Studies, Formalisms, and Structured Guidance in Computational Social Science Applications.
|
Project TIER Transparency and Reproducibility Speaker Series, Feb 7, 2020.
| Community Efforts Advancing
Reproducibility and Transparency in Data and
Computationally-Enabled Research.
|
ISE Seminar, University of Southern California, Jan 29, 2020.
| New Approaches to Reproducibility & Reliability of Research Results.
|
2019
|
CS Distinguished Seminar,
Northwestern University, Nov 20, 2019.
| The Lifecycle of Data Science: A Framework for Advancing Computational and Data-enabled Research.
| |
|
Center for Data and Computing Distinguished Speaker Series,
University of Chicago, Nov 19, 2019.
| Reproducibility is Not a Crisis. Now What? Next Steps for Advancing Computational and Data-enabled Science.
Video
|
Datapalooza 2019, UVA, Nov 1, 2019.
| Advancing Data Science as a Science: Paradigms, Practices, and Infrastructure
|
ASIS&T 82nd Annual Meetings, Oct 21, 2019.
| Open Access to Research Artifacts: Implementing the Next Generation Data Management Plan
|
Cognitive Systems Institute Group Speaker Series, Sept 26, 2019.
| Advancing Data-enabled Research via Reproducibility and Transparency
|
The National Academies of Science, Engineering, and Medicine,
Reproducibility and Replicability in Science: Next Steps Symposium, Washington D.C., Sept 24, 2019.
| Replicability: What It Means to Replicate a Study
|
Center for Informatics Research
in Science and Scholarship Seminar, University of Illinois at Urbana-Champaign, Sept 13, 2019.
| Advancing Data-enabled Discovery by Enabling Reproducibility and Transparency
|
Keynote: Parallel Computing 2019 Symposium
Tools and Infrastructure for Reproducibility in Data-Intensive Applications, Prague, Czech Republic, Sept 10, 2019.
| Advancing Reproducibility and Transparency in Data Inference Applications
|
Parallel Computing 2019 Symposium
Tools and Infrastructure for Reproducibility in Data-Intensive Applications, Prague, Czech Republic, Sept 10, 2019.
| Toward Enabling Reproducibility for Data-Intensive Research using the Whole Tale Platform
|
Trusted CI: The NSF Cybersecurity Center of Excellence, Indiana University, Aug 27, 2019.
| Reproducibility and Replicability in Cyberinfrastructure Systems
|
Friday Lunch Talks, Innovative Computing Laboratory, University of Tennessee - Knoxville, Aug 23, 2019.
| Advancing Reproducibility and Transparency via
Cyberinfrastructure and Community Standards
|
Space Telescope Science Institute, Baltimore, MD, July 17 2019. |
Reproducibility in Scientific Inference, Data
Dissemination, and Computational Environments
|
Reproducible Problem Solving with CyberGIS and Geospatial Data Science,
University of Illinois at Urbana-Champaign, July 9, 2019.
| Reproducibility in Computational and Data-enabled Science
|
2nd International Workshop on Practical Reproducible Evaluation of Computer Systems (P-RECS19),
Co-located with the 28th International Symposium on High-Performance Parallel and Distributed Computing (ACM HPDC 2019),
Phoenix, AZ, June 24, 2019.
| Continuous Integration Strategies in the Scientific Software Context
|
2nd International Workshop on Practical Reproducible Evaluation of Computer Systems (P-RECS19),
Co-located with the 28th International Symposium on High-Performance Parallel and Distributed Computing (ACM HPDC 2019), Phoenix, AZ, June 24, 2019.
| Implementing Computational Reproducibility in the Whole Tale Environment
|
Building Bridges Across the S&T Enterprise:
A White House National Science and Technology Council (NSTC) Conference,
Bethesda, MD, June 14, 2019.
| Data Infrastructure, Cyberinfrastructure and Reproducibility
|
Workshop on Education and Training for Reproducible Research - An Infrastructure Perspective,
NCSA, March 7, 2019.
| Cyberinfrastructure in the Classroom: A Case Study Using Whole Tale
|
2018
|
HPC in Cloud or Cloud in HPC: Myths, Misconceptions and Misinformation, Supercomputing18, Dallas, TX,
Nov 13, 2018.
| Scientific Reproducibility in the Cloud: Opportunities and Gaps
| |
|
Computational Reproducibility at Exascale 2018 (CRE2018), Supercomputing18, Dallas, TX,
Nov 11, 2018.
| Reproducibility in Computational and Data-enabled Science
|
The 1st Workshop on Reproducible, Customizable and Portable Workflows for HPC, Supercomputing18, Dallas, TX,
Nov 11, 2018.
| Assessing Reproducibility:
An Astrophysical Example of Computational Uncertainty in the HPC Context
|
Plenary: Ethics, law, and transparency. "Institute for the Secure Sharing of Online Data" (ISSOD) Workshop, Boston, MA, Nov 8, 2018.
| A Future of Research Transparency: Enabling Reproducibility in Repository Design
|
EPFL Symposium, Lausanne, Switzerland, Oct 25, 2018.
| Enabling Reproducibility in Computational and Data-enabled Science
|
Workshop II: HPC and Data Science for Scientific Discovery.
Part of the Long Program "Science at Extreme Scales: Where Big Data Meets Large-Scale Computing", Institute for Pure and Applied Mathematics, UCLA, Oct 19, 2018.
| Enabling Reproducibility in Computational and Data-enabled Science
|
CMU Open Science Symposium, Oct 18, 2018.
| Open Data, Code, and Computational Reproducibility
|
ProSem Seminar, iSchool, University of Illinois at Urbana-Champaign, Oct 12, 2018.
| How Open Licensing Can Support
Reproducibility in Computational Science
|
Law and Data Science Summit, University of Illinois at Urbana-Champaign, Oct 5, 2018.
| Legal Considerations in Data Science
|
Environmental Health Sciences Seminar, Yale School of Public Health, CT, Oct 3, 2018.
| Transparency and Reproducibility
in Evidence-based Policy Making
|
Biostatistics Seminar,
Northwestern University, IL, Oct 2, 2018.
| Infrastructure for Enabling Reproducibility in Computational and Data-enabled Science
|
Second Annual Junior Faculty Forum for Law and STEM,
Northwestern University School of Law, Chicago, Sept 28-29, 2018.
| Discussant: Scientific Knowledge Fraud
|
Whole Tale Workshop on Tools and Approaches for Publishing Reproducible Research,
Chicago, IL, Sept 13-14, 2018.
|
|
Keynote, R / Medicine,
Yale University, Sept 8, 2018.
| Computational Reproducibility in Medical Research: Toward Open Code and Data
|
Legal and Policy Issues Posed by Artificial Intelligence Advances,
Berkeley, CA, Sept 6, 2018.
| The Value of Computational Transparency
|
Rice University ECE Seminar,
Houston, TX, Sept 4, 2018.
| Advancing Data Science through a Lifecycle Approach
|
Joint Statistical Meetings, Big Data: Professional and Ethical Challenges from the Perspective of Actuaries, Statisticians and Data Scientists, Vancouver, BC, August 2, 2018.
| Two Key Ethical Challenges in Data Science
|
Joint Statistical Meetings, Introductory Overview Lecture: Reproducibility, Efficient Workflows, and Rich Environments, Vancouver, BC, July 31, 2018.
| How Computational Environments Can (Unexpectedly) Influence Statistical Findings
|
Joint Statistical Meetings, Transparency, Reproducibility and Replicability in Work with Social and Economic Data,
Vancouver, BC, July 29, 2018.
| Framing Reproducibility Issues in Computationally- and Data-Enabled Research
|
SciFOO Lightning Talk, Google X, June 23, 2018.
| The Reproducibility Debate
|
Research Reproducibility 2018: Building Research Integrity Through Reproducibility, University of Utah, June 15, 2018.
| Computational Reproducibility
|
Keynote, The 27th International Symposium on High-Performance Parallel and Distributed Computing, Arizona State University, June 13, 2018.
| Reproducibility in Computational and Data-enabled Science
|
First International Workshop on Practical Reproducible Evaluation of Computer Systems, Arizona State University, June 11, 2018.
| Enabling the Verification of Computational Results: An Empirical Evaluation of Computational Reproducibility,
and the Lightning Talk and our poster.
|
Keynote, IEEE Data Science Workshop, June 6, 2018.
| Reproducibility and Generalizability in Data-enabled Discovery, and our poster.
|
Symposium on Data Science and Statistics, May 18, 2018.
| Defining the AIM: An Abstraction for Improving Machine Learning Prediction
|
Publishing Summit 2018, Pacific Northwest National Labs, May 8, 2018.
| Reproducibility at Scale
|
Rice University, April 25, 2018.
| Paradigms for Data Science: Reproducibility, Transparency, and Generalizability
|
Reproducibility Symposium, School of Public Health, Yale University, April 16, 2018.
| Computational Reproducibility
|
IEEE Panel of Editors Meeting: Research Reproducibility, April 13, 2018.
| Stakeholder Roles in Enabling Reproducibility in Computational Research
|
Keynote, Northwestern Computational Research Day, April 10, 2018.
| Reproducibility in Computational Research: Code, Data, Statistics, and Implementation
|
Computer & Information Sciences Distinguished Speaker Lecture,
University of Delaware, April 6, 2018.
| The Science of Computational Reproducibility
|
SOS-22 Workshop: HPC and Data Science, Waikoloa, Hawaii, March 28, 2018.
| Reproducibility at Exascale
|
Roundtable on Data Science Postsecondary Education, Berkeley, CA, March 23, 2018.
| Data Science as a Science: Methods and Tools at the Intersection of Data Science and
Reproducibility
|
Guest Lecture, UC Berkeley, CA, March 22, 2018.
| Knowledge and Information vs Data
|
Towards New Principles for Enhanced Access to Public Data for Science, Technology and Innovation;
OECD Committee for Scientific and Technological Policy Workshop, Paris, March 13, 2018.
| Enabling Access to Research Data: Key Steps Forward
|
Barnard College at Columbia University, NYC, Feb 8, 2018.
| Computational Knowledge and the Scholarly Record
|
Social Science Foo Camp Lightning Talk, Facebook, Menlo Park, CA, Feb 3, 2018.
| What's a Finding?
|
Research Panel Presentation, Learning from the Science of Cognition and Perception for Decision-making: A Workshop,
Decadal Survey of Social and Behavioral Sciences for Applications to
National Security, (or here), Washington DC, Jan 24, 2018
| Trust in Research Findings
|
Invited Public Seminar, Department of Statistical Sciences and the iSchool, University of Toronto, Jan 18, 2018
| The Meaning of Evidence in Computational and Data-Driven Science
|
2017
|
Assessing the Availability of Source Code in Computational Physics, SC17, Nov 14-16, 2017
| Assessing the Availability of Source Code in Computational Physics
| |
|
Reproducibility and Uncertainty in High Performance Computing, SC17, Nov 14, 2017
| Reproducibility and Uncertainty in High Performance Computing
|
Seminar in the Department of
Statistical Science, Cornell University, Nov 1, 2017
| Structuring Machine Learning in Data Driven Science
|
Seminar in the Department of
Statistical Science, University of Toronto, Oct 26, 2017
| Structuring Research for Reproducibility
in Data Science Investigation
|
Statistics Department Seminar, Northwestern University, Oct 18, 2017
| CompareML: Structuring Machine Learning
Research in Data Driven Science
|
Epidemiology Seminar, Stanford University School of Medicine, Oct 17, 2017
| Reproducibility in the Biomedical
Sciences: Progress, Policy, and Prognosis
|
Guest Lecture, Stanford University Stats Department, Oct 16, 2017
| Reproducibility in Computational Science (video to come)
|
Open Science Evening Talks 2017, EPFL, Lausanne, CH, Sept 28, 2017
| A Computable Scholarly Record (video here)
|
Open Science in Practice, EPFL, Lausanne, CH, Sept 28, 2017
| Software and Scientific Reproducibility
|
Featured Speaker, Code Ocean Workshop: Effective Practices in Computational Reproducibility, Columbia University, NYC, Sept 19, 2017
| What You Need to Know About Reproducibility
|
Keynote, Research Data Management Implementations (RDMI) Workshop, Arlington, VA, Sept 14, 2017
| Research Data Management Implementations:
Towards the Reproducibility of Science
|
Keynote, Chameleon User Meeting, Argonne National Lab, Lemont, IL, Sept 13, 2017
| The Role of Cyberinfrastructure in Reproducible Science
|
European Alpbach Forum,
Technology Symposium, Alpbach, Austria, August 25, 2017
| Knowledge and Understanding in the Age of Data
|
National Science Foundation,
Workshop on Reproducibility Taxonomies for Computing and Computational Science,
Arlington VA, July 25, 2017
| Reproducibility Background: A Historical Perspective
|
National Academies of Science, Engineering and Medicine, Toward an Open Science Enterprise,
Washington DC, July 20, 2017
| Enhancing Reproducibility for Computational Methods
|
Math+Stats+X A
Conference on the Occasion of Dave Donoho's 60th Birthday,
Stanford University, June 25, 2017
| 25 Years of Reproducibility
|
Dataverse Community Meeting 2017,
Harvard University, June 16, 2017
| Toward a Reproducible Scholarly Record
|
Summer@Census Seminar, Census Bureau,
Suitland, MD, June 5, 2017
| Implementing Reproducible
Computational Research:
Policy and Practice
|
METRICS Seminar,
Stanford, CA, May 15, 2017
| Reproducibility in Computationally-Enabled Research: Integrating Tools and Skills
|
Data Science @ HEP,
Fermilab, Batavia, IL, May 9, 2017
| MLCompare: A Facilitating Framework for Machine Learning Research
|
AERA Annual Meeting,
Data Sharing and Research Transparency at the Stage of Journal Publishing,
San Antonio, TX, April 30, 2017
| Publishing Computational Findings
|
AERA Annual Meeting,
Using Big Data: The Ethics, Dilemmas, and Possibilities for Educational Opportunity,
San Antonio, TX, April 28, 2017
| Law & Ethics of Big Data Research Dissemination
|
The Judith Resnik Year of Women in ECE Seminar,
Carnegie Mellon University, Pittsburgh, PA, April 6, 2017
| Reproducibility in Computationally-enabled Research
|
National Academy of Sciences, Arthur M. Sackler Colloquia,
Reproducibility of Research: Issues and Proposed Remedies,
Washington D.C., March 9, 2017
| Data Sharing and Reproducibility
|
National Academy of Sciences, Arthur M. Sackler Colloquia,
Reproducibility of Research: Issues and Proposed Remedies,
Washington D.C., March 8, 2017
| Framing the Issues: Reproducibility in Many Forms
|
Institute for Advanced Computational Science,
Stony Brook University, March 2, 2017
| Implementing Reproducible Computationally-Enabled Science
|
Heidelberg Institute for Theoretical Studies (HITS) Colloquium,
Heidelberg, Germany, February 20, 2017
| Implementing Reproducibility in Computational Science
|
USGS, Community for Data Integration monthly meeting,
Washington DC, February 8, 2017
| Facilitating Reproducibility of Scientific Findings through Access to Data, Code, and Research Objects
|
IST Lunch Bunch,
Caltech, January 10, 2017
| Enabling Reproducibility in Computational Science
|
Envisioning the Scientific Paper of the Future,
Caltech, January 9, 2017
| Imagining a Reproducible Scholarly Record
|
2016
|
AGU Fall Meeting, Reproducible Research in Geosciences with
Emphasis on Provenance of Information as an Essential Component I,
San Francisco, Dec 15, 2016
| Introducing the Whole Tale Project: Merging Science and Cyberinfrastructure Pathways
| |
|
A University Symposium: Promoting Credibility, Reproducibility, and Integrity in Research,
Columbia University, NYC, Dec 9, 2016
| Deconstructing Reproducibility
|
IC Colloquium,
Ecole polytechnique fédérale de Lausanne, Switzerland, Nov 17, 2016
| Imagining a Reproducible Scholarly Record
|
Seance de reflexion of the Swiss National Research Council on "2050: A Science Odyssey," Interlaken, Switzerland, Nov 16, 2016
| Science: Set the Default to Open
|
IEEE Workshop on The Future of Research Curation and Research Reproducibility, Washington, DC, Nov 6, 2016
| Economics of Reproducibility
|
Center for Research Computing Seminar, Notredame University, Oct 24, 2016
| Reproducibility in Computational Science: A Computable Scholarly Record (video)
|
Invited Presentation, NSF GEO Directorate Advisory Committee Meeting, Washington, DC., Oct 19, 2016
| Reproducibility in Computational Science: Framing and Steps Forward
|
RDI2 Workshop: Reproducibility in Experimental and Computational Science Research - A Rutgers 250 Event,
Rutgers University, Oct 10, 2016
| Toward Really Reproducible Research: Policies and Practices
|
Bringing Rigor and Reproducibility to Research: A Symposium for the Health Sciences,
University of Pittsburgh, Sept 22, 2016
| Rethinking the Scholarly Record:
Facilitating Reproducibility of Computational Results
|
RDA 8th Plenary Joint meeting:
IG Active Data Management Plans,
IG Preservation e-Infrastructure and IG Reproducibility, Denver CO, Sept 17, 2016
| Supporting Public Access to Supplemental Scholarly Products Generated from Grant Funded Research
|
IG Reproducibility RDA 8th Plenary meeting, Denver CO, Sept 16, 2016
| Reproducibility Interest Group
|
International Data Forum, Data for the Public Good: Responsibilities,
Opportunities and Dangers in a Data-Aware Society,
International Data Week, Denver CO, Sept 14, 2016
| Scientific Transparency, Integrity, and Reproducibility
|
Colloquium on Future Cyberinfrastructure for Scientific Discovery,
Computer Science and Telecommunications Board,
The National Academies of Sciences, Engineering, and Medicine,
Washington DC, Sept 8, 2016
| Achieving Reproducibility in Computationally Enabled Research
|
CIRSS Seminar, University of Illinois at Urbana-Champaign, Sept 2, 2016
| Introducing the Whole Tale Project
|
Workshop on the Growth of Computer Science Undergraduate Enrollments,
NAS, Washington DC, August 15, 2016
| Dimensions of Computing:
A Data Science Perspective
|
Intellectual Property Scholars Conference, Stanford Law School, CA, August 11, 2016
| Disclosing Data Enabled Research: Intellectual Property Policy for Publishers, Technology Transfer Offices, and Librarians
|
Keynote Talk, NSF Workshop on Geospatial Data Science, NCSA, IL, July 25, 2016
| Reproducibility in Geospatial Sciences and Technologies
|
Invited Talk,
SIAM Annual Meeting, Boston, MA, July 14, 2016
| Implementing Reproducibility in Computational Science (video)
|
Keynote, María de Maeztu Annual Event: Data-driven Knowledge Extraction Workshop,
Universitat Pompeu Fabra, Barcelona, June 29, 2016
| Reproducibility in Computational Research
(video)
|
AGU DMM Advisory Board Meeting, Washington DC, May 26, 2016
| Software and Reproducible Research
|
2nd ACM Workshop on Data, Software, and Reproducibility in Publication,
New York City, NY, May 4, 2016
| Legal Issues in Reproducible Research
|
Keynote, Coalition for Networked Information Annual Meeting,
San Antonio, TX, April 4, 2016
| Defining the Scholarly Record for Computational Research (video)
|
Learning Tools to Promote Reproducible Research and Open Science, Chicago Chapter of the American Statistical Association,
Northwestern University, Mar 31, 2016
| A CyberInfrastructure Wish List for Statistical and Data Driven Discovery
|
Data Summit, University of Illinois at Urbana-Champaign, Feb 5, 2016
| Reliable Inference in Data Science: Why Should We Care?
|
2015
|
Social and Decision Analytics Laboratory Seminar, Virginia Tech, Arlington, VA, Dec 2, 2015
| The Importance of Scientific Reproducibility in Evidence-based Rulemaking.
| |
|
Cyberinfrastructure (CI) for NSF Large Facilities Workshop,
Arlington, VA, Dec 1, 2015
| Data Access and Ownership.
|
Invited talk, METRICS Conference,
Stanford University, CA, Nov 19, 2015
| Reproducibility in Computational Science.
|
Invited Plenary, SC15, Austin, TX, Nov 18, 2015
| Reproducibility in High Performance Computing (video here).
|
Closing Remarks, Data Science @ LHC 2015 Workshop,
Open Data & Roundtable on Data Access, CERN, Geneva, Nov 13, 2015
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |