A multi-modal approach to measuring the effect of XAI on air traffic controller trust during off-nominal runway exits

Abstract

Lack of transparency has demonstrated to be a stumbling block in building Air Traffic Controller (ATCO) trust towards intelligent decision aids. To address this issue, a runway exit prediction decision aid, with explainability, was developed with the trait of providing explanations involving the top three contributing features to its predictions. In order to evaluate the impact of its explanations on ATCO Trust, this decision aid was used in a human-in-the-loop study in the context of off-nominal scenarios, with 12 participants and a total of 67 trials involving aircraft on final approach. A multi modal approach was adopted with three types of data (questionnaire, behavioural, physiological) being collected in this study to ensure a more complete understanding of the effects of explainability on ATCO trust. The results indicated that higher levels of perceived transparency led to an increase in trust levels, with an accompanied increase in cognitive load and complacency, even with low prediction accuracy by the intelligent decision aid. As such, these effects must be accounted for in designing XAI decision aids for off-nominal events, when attempting to enhance trust levels by increasing transparency.