QLS Seminar Series - Roger Peng

Lessons in Disaster: How Can We Learn From Data Analysis Failures?

Roger Peng, Johns Hopkins University
Tuesday November 10, 12-1pm
Zoom Link: https://mcgill.zoom.us/j/91589192037

Abstract: The details of data analysis failures are often not publicly observable, making it difficult to develop a systematic understanding of how failures occur and how they can be prevented in the future. Most published analyses barely provide sufficient details of successful analyses, much less any failures. Reproducible research is a concept that has been promoted as a way to increase transparency in analysis, however it is not clear what value it has in preventing failure. I will describe methodology that we can employ to develop generalizable knowledge about data analytic failures and allow others to learn from unexpected outcomes. I will also discuss some case studies of widely publicized data analysis failures and attempt to distill what, if any, lessons we can take from these cases to improve future analyses.