Neural Human-System Interface (NHSI) Technology

Scott Makeig, Tzyy-Ping Jung, and colleagues

There is rapidly growing interest in the development of intelligent interfaces in which operator state information derived from psychophysiological and/or video-based measures of the human operator is used directly to inform, interact with, or control computer-based systems. Adequate signal processing power is now available at reasonable cost to implement in near-real time a wide range of spectral, neural network, and dynamic systems algorithms for extracting information about psychological state or intent from multidimensional EEG signals, video images of the eyes and face, and other psychophysiological and/or behavioral data. Aspects of cognitive state that might be monitored using NHSI technology include alertness, attention, intention, readiness for duty, and emotion.

In our increasingly automated society, the role of human operators of complex system tend towards information or work overload or underload. In many environments, for much of their work time, some operators do little more than monitor automated system activity, while needing to remain prepared to deal quickly and accurately with complex combinations of circumstances.

In other work environments, cognitive workload may threaten to overwhelm operators' attentional capabilities. Neural human-systems interfaces for monitoring and helping manage operator alertness and attention could improve reliability and performance in such environments.

Active NHSI research, in Amierca, Europe, and elsewhere focuses on brain-actuated control for quadraplegics and high-workload environments. Video-based eye tracking may have many uses, including brain/eye-actuated control and attention monitoring. Video-based NHSI emotion monitoring might find unforeseen applications, including interactive training.

However, formidable obstacles to practical implementation of NHSI technology remain, including questions of individual differences, measure extraction and stability, noise management, and habitability. New designs for convenient silicon-based dry EEG electrodes and miniaturized infrared video cameras may solve some problems, but all are presently controversial.

Alertness Monitoring

For more information on NHSI research, particularly in the area of alertness monitoring, visit the home page of my colleague Tzyy-Ping Jung or contact me by email,

About this alertness monitoring session.

There was a NIPS95 Workshop on prospects for NHSI technology, Dec. 2, 1995, in Vail, Colorado. For more information, see the NHSI workshop home page.

Scott Makeig