Get In Touch
Menu

OWL2

On WorkLoad Measures (OWL) 2

Scroll Down

Who We Are

Project Participants

About

This is the homepage of the project On WorkLoad Measures (OWL) 2 funded by Trafikverket.

Being able to balance air traffic controllers' (ATCOs’) workload (WL) at a moderate level is considered a vital safety barrier that currently depends on operators' subjective self-estimation. However, target fixation (such as attentional tunneling, which describes the neglect of other critical stimuli) and mental fatigue are known phenomena that reduce operators' ability to accurately estimate their own WL. Being able to support the self-estimation based on empirical measurements would set the basis for a WL monitoring system. In the ongoing research project OWL (On Workload Measures) part 1, a simulation study was conducted with 18 ATCOs with varying taskload scenarios. We collected (in addition to subjective WL with numerical scales) objective, non-intrusive indicators of workload: eye movement measurement/eye observation (ocular) and audio recording. Reference WL indicators were EEG (electroencephalography) and self-reports using the adapted Cooper-Harper scale. To analyse the data, machine learning (ML) has been used to predict workload from eye movements with good success. We achieved 96% accuracy and an F1 score of 84%. However, incorrect WL predictions could lead to bad operational decisions (false prediction risk), particularly as operators may overly trust the monitoring system due to automation bias. A key challenge is that the conditions required for maintaining high accuracy are not fully understood. It is therefore unclear how reliable WL predictions truly are if local and temporal conditions vary unforeseen, which lead to prediction uncertainty. This poses a risk in safety-critical applications that depend on constantly accurate predictions over time. Now, building on the findings from OWL 1, OWL 2 aims to better understand the relationship between objective indicators and the ability to predict WL using non-invasive methods. The primary focus will be on identifying and analyzing sources of uncertainty (uncertainty factors) using sensitivity analysis to examine how they affect prediction accuracy. For example, it remains unclear how accurately WL can be predicted for individuals not being included in the ML training dataset. We consider individual differences as a predominant uncertainty factor. Additionally, the lower F1 score suggests that the model might make more errors when predicting high WL levels (minority class). A secondary focus will be on uncertainty mitigation strategies by, e.g., optimizing indicators used for prediction. This involves also an extension to alternative physiological and behavior-based WL indicators, including operator voice, visual scan pattern and heart measurement. The analysis will use the existing data collected in the scope of OWL 1, which is suitable for the objectives of OWL 2.

Publications from OWL 2


Publications from OWL 1


A. Lemetti, L. Meyer, M. Peukert, T. Polishchuk, C. Schmidt, H. Alpfjord Wylde:
Predicting Air Traffic Controller Workload from Eye-Tracking Data with Machine Learning,
Submitted for Publication


A. Lemetti, L. Meyer, M. Peukert, T. Polishchuk, C. Schmidt, H. Alpfjord Wylde:
Predicting Air Traffic Controller Workload using Machine Learning with a Reduced Set of Eye-Tracking Features,
In The 35th European Association for Aviation Psychology (EAAP) Conference


A. Lemetti, L. Meyer, M. Peukert, T. Polishchuk, C. Schmidt, H. Alpfjord Wylde:
Eye in the Sky: Predicting Air Traffic Controller Workload through Eye-tracking based Machine Learning,
In 43rd Digital Avionics Systems Conference (DASC) 2024


A. Lemetti, L. Meyer, M. Peukert, T. Polishchuk, C. Schmidt:
Discrete-Fourier-Transform-Based Evaluation of Physiological Measures as Workload and Fatigue Indicators,
In 42nd Digital Avionics Systems Conference (DASC) 2023
[PDF], [Presentation slides by L. Meyer]

News