Modelling Fatigue in GazeTalk using Text Input, Subjective Ratings, and Saccade Probing
Fatigue is a key limiting factor in gaze-based communication systems such as GazeTalk. While previous work has shown that interaction-based features (e.g., typing speed, pauses, and error rates) can be used to estimate fatigue, there remains a need to integrate multiple complementary signals into a unified and deployable model. In particular, combining behavioral text input, subjective self-reports, and lightweight eye-movement probes may provide a robust and clinically relevant estimate of patient fatigue. Recent work has demonstrated that gaze-based typing behavior reflects fatigue-related changes in performance, and that prosaccades offer a low-intrusive method for probing cognitive load through latency measures.
The goal of this project is to design and implement a multimodal fatigue model for GazeTalk that integrates these signals into a real-time adaptive framework. More specifically, the project should:
- Review literature on fatigue modelling in HCI, AAC systems, and eye-tracking-based workload assessment
- Extend the existing GazeTalk data pipeline to include synchronized logging of text input features, subjective fatigue ratings, and saccade probe responses
- Design and implement a lightweight saccade probing mechanism embedded in the interface
- Construct a dataset combining behavioral features (e.g., WPM, pauses, error rate), subjective ratings (e.g., Likert-scale), and saccade latency measures
- Train and evaluate machine learning models for fatigue prediction (e.g., regression or classification models)
- Explore how the fatigue model can be used to drive adaptive system behavior (e.g., interface simplification, pacing, or break suggestions)
This project builds directly on recent work within the GazeTalk system, including the existing fatigue modelling pipeline based on interaction data and the integration of lightweight saccade probes for workload estimation. The student will work with a functioning prototype and contribute to the development of a clinically relevant fatigue-aware interaction model. The project will be supervised by postdoctoral researcher Ekky Tammarar Alfian and co-supervised by Prof. John Paulin Hansen. This project can be formulated into a Master thesis/Bachelor thesis/student project. The project is expected to start in the Fall semester 2026 (negotiable).
Thesis type: Machine learning-based modelling and system integration
Technical skills: Data processing and feature engineering; machine learning; frontend/backend integration
Research skills: Human-Computer Interaction; Machine Learning; Cognitive modelling
Educational background: Computer Science, Artificial Intelligence, Human-Centered Computing
ECTS credits: MSc thesis / BSc thesis / Special course (5 or 10 ECTS)
Contact: < email supprimé pour raison de sécurité >
By working on the project, you will be part of a team aiming to improve lives for stroke patients!