Temporal Attentional Bias and Its Effects on Affective Experience in Multimodal Interaction Interfaces of Intelligent Cockpits: An Interdisciplinary Study Based on Behavioral and Eye-Tracking Evidence

Abstract

Abstract

Background and Gaps: With the widespread adoption of intelligent connected vehicles, the intelligent cockpit—serving as the core carrier of human–machine interaction—has drawn increasing attention in terms of affective interface design. Existing studies primarily focus on the effects of spatial layout and information presentation in visual and auditory interfaces on drivers’ emotions, while largely neglecting the role of temporal attention allocation (i.e., temporal attentional bias) in the perception of multimodal affective feedback. In complex driving scenarios, discrepancies between the expected timing and actual presentation of interaction feedback may significantly influence users’ affective experience and cognitive load.

Methods: From the perspective of interdisciplinary design innovation, this study integrates temporal attention theory from cognitive psychology with human–computer interaction design to develop a multimodal (visual and auditory) affective feedback experiment. By manipulating the expected interval of interface feedback (short vs. long) and cue validity (valid vs. invalid), temporal attentional bias was induced. Participants (n = 105) were assessed in terms of reaction time (RT), accuracy, and eye-movement trajectories when responding to interface feedback with different emotional valences (positive vs. negative).

Experimental Implementation: The experiment was conducted using a customized simulated driving and intelligent cockpit interaction platform. Eye-tracking devices and behavioral recording software were employed for synchronized data acquisition. Data analysis was performed using generalized linear mixed models (GLMM) to examine the interaction effects among temporal cues, expected intervals, and emotional valence. Additionally, hidden Markov models (HMM) were applied to analyze eye-movement sequence characteristics.

Key Findings: The results indicate that temporal attentional bias plays a significant moderating role in the perception of negative affective feedback, particularly under short expected interval conditions. Specifically, when negative feedback appears at an unexpected time (invalid cue), participants exhibit significantly increased reaction times and decreased accuracy. In contrast, the perception of positive feedback demonstrates strong robustness to temporal attentional bias. Furthermore, eye-tracking data reveal that temporal attentional bias leads to prolonged fixation durations on negative feedback, accompanied by increased cognitive load.

Significance and Contributions: This study is the first to introduce temporal attention mechanisms into the affective interaction design of intelligent cockpits, revealing an asymmetric effect of temporal expectation in multimodal affective feedback perception. The findings provide a theoretical foundation for temporal flow design in intelligent product interfaces and offer practical guidance for designers to optimize feedback timing, thereby alleviating drivers’ cognitive burden in complex scenarios and enhancing both the overall affective experience and safety of intelligent cockpits.

Keywords: Intelligent cockpit; temporal attentional bias; affective design; multimodal interaction; cognitive load