I Spy with my AI: cognitive ergonomics of CCTV control room operators during new human-computer interactions – an ExperienceTwin application
March 30th, 2023
Public space in our cities is being monitored by more and more security cameras, which increases the cognitive demand for closed-circuit television (CCTV) operators. Nowadays, AI-driven support systems using computer vision algorithms can be used to assist human operators in their surveillance tasks. However, to develop intuitive, AI-driven interfaces and evaluate their impact on operator performance, extensive user testing is required.
As part of the imec.icon project SenseCity, we conducted a study in which we designed a standardised procedure to evaluate the impact of new (prototypes of) support systems in CCTV control rooms on operator performance and cognitive load in virtual reality (VR). Cognitive load is an important aspect to keep in mind when developing support systems as they should not only increase accuracy but also avoid operator overload. In a later stage, we tested a prototype of an AI-driven support system that uses visual cueing techniques and evaluated its impact on user performance and cognitive load. Importantly, we assessed this under varying conditions of work demands, which subjected the operators to different levels of cognitive load. In this way, we investigated whether the addition of assistive technology has different effects on cognitive load during distinct scenarios. For example, assistive technology might have a significantly different impact on an operator during day-to-day operations compared to when managing a crisis.
Notably, we took an innovative multimodal approach that allowed us to cross-validate and triangulate data from different sources, enhancing the reliability and validity of our findings. More specifically, we measured a host of different objective and subjective markers using eye tracking, electroencephalography (EEG) and questionnaires.
As promising as the multimodal approach sounded, we needed the correct environment in which we could evaluate it and test prototypes that were not fully operational. VR provided the perfect set-up to do so as it presented us with a controlled testing environment in which we could alter various factors, such as lighting conditions and background noise. VR simulators also offer the ability to construct and evaluate operator support systems in a shorter amount of time, without the need for fully operational support systems and without the need to impede ongoing control room operations. Additionally, VR allowed us to record all interactions with the virtual environment, providing a rich data set that described the participants’ behaviour and cognitive processes.
The current study showed that the tested support system improved the operators' performance and decreased their cognitive load irrespective of the levels of induced cognitive load. Furthermore, we demonstrated that VR is a valuable medium for assessing existing and future operator support systems in CCTV surveillance rooms. The results also underlined that future research should not merely evaluate the effectiveness of AI algorithms embedded in operator support systems, but also incorporate the UX and usability perspective. This way, the full human-computer interaction and the system design can be evaluated alongside the effectiveness of the algorithm itself. The presented methodology can also be applied to different types of control rooms, for example, in nuclear power plants, petrochemical plants or in air traffic control towers.
Want to know more? Check out our scientific article published in the International Journal of Industrial Ergonomics or get in touch!