On the problem statement and evaluation of automatic software repair

What: On the problem statement and evaluation of automatic software repair
Who: Martin Monperrus (University of Lille & Inria)
When: May 28th, 2014, 11:00 – 12:00
Where: Inria Lille, Room B21, http://goo.gl/maps/32z7m

Abstract: At ICSE’2013, there was the first session ever dedicated to automatic program repair. In this session, Kim et al. presented PAR, a novel template-based approach for fixing Java bugs. We strongly disagree with key points of this paper and discuss them in this talk. First, we aim at explaining why we disagree with Kim and colleagues and why the reasons behind this disagreement are important for research on automatic software repair in general. Second, we aim at contributing to the field with a clarification of the essential ideas behind automatic software repair. In particular we discuss the main evaluation criteria of automatic software repair: understandability, correctness and completeness. We show that depending on how one sets up the repair scenario, the evaluation goals may be contradictory.