Presented at the 44th Annual Society for Neuroscience Conference, Washington, DC(November 2014)

Biometric techniques for sensing peripheral nervous system indicators (e.g., heart rate, blood pressure, and galvanic skin response), when analyzed and interpreted correctly, provide an opportunity to objectively assess and forecast cognitive, physical, and emotional human states. However, the accurate interpretation of biometric data depends on the environment in which the individual is performing and on the type of task the individual is undertaking. The effort described herein focused on (1) proving the feasibility of an approach to acquire, model, and interpret cognitive, emotional, and physical state from physiological data, and (2) forecasting the effect that state will have on cognitive performance. We first identified a set of inexpensive, commercially available, off-the-shelf sensors that are simple to use, robust enough to use in real-world environments, and provide access to the raw data. We used a Zephyr Bioharness to measure heart rate, breathing rate, skin temperature, and posture; a body-mounted 9-axis accelerometer, including a 3-axis gyroscope (to measure orientation), a 3-axis accelerometer, and a 3-axis magnetometer (to measure the direction of the Earth’s ferromagnetic axis); an identical ship-mounted 9-axis accelerometer for collecting environmental motion information from the boat; and a mobile device (a 10-inch Samsung Galaxy tablet) to deliver cognitive tasks. Cognitive tasks included a working memory task (dual n-back task; (Jaegi et al., 2003), a problem solving task (basic arithmetic), and an inhibition task (Stroop task; (Stroop, 1935)). Once the data were collected, we used complex event processing methods to extract and fuse the best combination of indicators of motion sickness and cognitive performance based on previous findings in the literature and subject matter expert input. Next, we modeled the fused data with a hybrid computational approach to combine different types of indicators to infer symptoms of fatigue, mood, sickness, and motivation directly from data. We then used the modeled data to forecast cognitive degradation due to motion exposure before the individual subjectively reported her symptoms. We present the results of our pilot experiment in which we demonstrated that we could feasibly anticipate motion exposure effects on cognition for a small subject population. We believe this approach can be used across a wide range of sensing devices, assessment environments, and human states of interest.

¹ Charles River Analytics
² Brandeis University