PhD defence of Raihan Seraj – Dynamic modeling of mental workload and fatigue-aware decision referrals in human-automation teams

Abstract

Human automation teams are increasingly prevalent in domains such as aviation, autonomous vehicles, healthcare, and manufacturing. In such systems, the cognitive states of humans, including mental workload and fatigue, play a pivotal role in determining the joint performance of the team.

In this thesis we focus on the dynamic nature of the human cognitive state, which exhibits temporal correlation. Subjective measures of mental workload can be obtained using standard questionnaires like the NASA-TLX, however, their administration is often impractical as it interferes with the primary tasks of the human operator. Therefore, it is of interest to estimate these subjective measures from less intrusive observations. Evidence suggests that mental workload is a dynamic process, so incorporating historical measurements could reduce its estimation error. Additionally, the estimation of operator performance in human automation teams is essential in optimizing task effectiveness and facilitating efficient resource allocation. We present and compare different dynamic schemes to estimate an operator’s performance on classification tasks, i.e., classification accuracy, and her subjective ratings on subscales of the NASA-TLX questionnaire, which measure mental workload across multiple dimensions. These schemes differ in the information available for estimation. We test these schemes on data collected from a scenario where a human and an automation perform a series of classification tasks for simulated mobile objects. Our analysis of the interaction data and the estimation schemes indicates that employing dynamic estimation leads to decreased estimation errors for certain NASA-TLX subscale ratings.

We next address the problem of allocating decision tasks, via decision referrals from the automation to the human, in a context where human performance depends on workload and a dynamically evolving level of fatigue. To model this, we formulate the decision referral problem as a MDP, allowing us to explicitly account for the changing state of human fatigue and its impact on decision-making performance. To solve the MDP, we propose an approximate dynamic programming approach, which yields a fatigue-aware decision referral policy. We evaluate the performance of this policy, which considers fatigue dynamics, against a task deferral policy that bases its decisions solely on instantaneous fatigue levels. We then analyze the robustness of the fatigue-aware policy.