Área: Engenharia Informática e de Computadores

Despacho de nomeação de Júri

Título da Tese: MENTOR : Automated Feedback for Introductory Programming Exercises

Local da Prova: Anfiteatro PA-3 (Piso -1 do Pavilhão de Matemática) do IST

Data: 10/04/2025

Hora: 14h00
Abstract: The increasing demand for programming education has given rise to all kinds of online evaluations such as Massive Open Online Courses (MOOCs) focused on introductory programming assignments (IPAs). As a consequence of a large number of enrolled students, one of the main challenges in these courses is to provide valuable and personalized feedback to students. This thesis presents MENTOR, a semantic automated program repair (APR) framework designed to provide Automated Feedback for Introductory Programming Exercises. MENTOR addresses this challenge by generating possible repairs for faulty student programs, enabling semantic repairs validated through execution on a test suite and by highlighting these faulty statements to the students. Hence, in the context of this work, we provide scientific contributions in several areas, such as program clustering and analysis, automated fault localization and program repair. MENTOR advances the state of the art in the referred areas and provides an innovative practical framework to be deployed in educational environments. Current state-of-the-art symbolic repair tools, Clara and Verifix, are limited to correcting programs using correct implementations with an identical control flow graph (CFG) to the faulty program. In contrast, MENTOR leverages a clustering-based approach that utilizes a broader set of previous successful submissions, allowing it to repair incorrect programs without strict structural alignment. Additionally, MENTOR incorporates a variable alignment module using Graph Neural Networks (GNNs) to effectively map variables between programs, enhancing the accuracy of repairs. MENTOR's fault localization module, CFaults, employs MaxSAT-based techniques to precisely identify buggy code segments. Coupled with the program fixer module that integrates Formal Methods (FM) and Large Language Models (LLMs) through a Counterexample Guided Inductive Synthesis (CEGIS) loop, MENTOR iteratively refines programs by synthesizing patches to the faulty statements identified by CFaults. Furthermore, this work also proposes a language-agnostic automated assessment tool, GitSEED, that enhances student learning by providing personalized feedback on code submissions and successfully integrates CFaults for effective fault detection on student code. Experimental results on C-Pack-IPAs demonstrate that MENTOR significantly improves repair success rates, achieving 64.4%, compared to just 6.3% for Verifix and 34.6% for Clara.