Peng Kuang's PhD defence

Thesis title: Building Gaze-aware Programming Environments

Author: Peng Kuang, Department of Computer Science, Lund University

Faculty opponent: Associate Professor Andrew Begel, Carnegie Mellon University, USA

Examination Committee:

• Professor Yvonne Dittrich, IT University of Copenhagen, Denmark
• Professor Veronica Sundstedt, Blekinge Institute of Technology
• Professor Emilia Mendes, Aarhus University, Denmark
• Deputy: Postdoc Sara Willhammar, Lund University

Session chair: Senior Lecturer Michael Doggett, Lund University

Supervisors:

• Main supervisor: Senior lecturer Emma Söderberg, Lund University
• Professor Martin Höst, Malmö University
• Researcher Diederick C Niehorster, Lund University

Location: E:1406, E-huset, Klas Anshelms väg 10/Ole Römers väg 3, Lund

Here is a link to follow the defence via Zoom

Here is a link to download the thesis at LU Research Portal

Abstract

Programming is a cognitively demanding exercise. Artificial Intelligence (AI) as a disruptive technology is redefining the practice of programming and transforming software engineering. As AI is evolving to a multimodal version that accommo- dates not only texts but also speech, images, and more, we see an opportunity to design eye-tracking based assistance to support programmers. Since AI has taken the heavy lifting of producing code, we speculate that programmers will read and understand a larger amount of code and thereby spend more of their time read- ing it. We deem this a promising problem domain where eye-tracking can be of assistance.
To explore this inquiry, we undertook two mapping studies to establish the problem and solution constructs. We then surveyed 68 professional developers to understand this representative cohort and gather concrete, situated problems from them. After that, we co-developed multiple versions of design artifacts with a variety of groups of programmers. Finally, we employed a mixed-methods approach, including a pre-experiment survey, a controlled experiment, and post-experiment interviews with 40 novice programmers, to evaluate the proof-of-concept GazePrinter.
From the first study, we found that eye-tracking so far is used mostly for education-oriented studies in the research community focused on software devel- opment. There is a need to bring it closer to practitioners. From the second study, we identify that the gaze data produced by eye trackers has been explored with a collection of machine learning techniques. However, these models were trained with small samples that might carry bias and insufficiency. Contemporary ma- chine learning techniques may be able to compensate for that. From the survey, we learned that developers have already adopted AI assistance, and they are mostly positive about it despite room for greater accuracy and capability. As eye-tracking is relatively novel to them, most developers are unsure about how it can help them. From the design study, we realized that programmers are intrigued by gaze-based assistance in a programming environment. Lastly, from the evaluation study, we found that our designed intervention, GazePrinter, can nudge novice programmers to read code in a manner that more closely aligns with experts.
For future work, we invite research exploring adaptive, gaze-driven assistance and interactions in AI native environments for different programmers.