Presented at the International Neuroergonomics Conference, Philadelphia, PA (June 2018)

Background

Assessing cognitive workload using functional near-infrared spectroscopy (fNIRS) in labs is well established. Increased workload corresponds with increase in prefrontal blood oxygenation (HbO₂) correlated with increased task engagement. Once the task becomes too difficult, HbO₂ decreases as does task engagement and performance (Ayaz, et al., 2012).  However, fNIRS sensors useful for assessing cognitive workload during normal activities in real-world environments are only recently emerging. Standard sensors are large (e.g., full-head), expensive (~$10K) and require heavy equipment (e.g., batteries, laptops).

Under an Army-funded effort (MEDIC), we designed a portable, cost-effective fNIRS sensor. Under a NASA-funded effort (CAPT PICARD), we validated our sensor against the NINScan developed at Massachusetts General Hospital. We used a gold-standard task known to affect cognitive workload (n-back; (Kirchner, 1958)) and a more complex multi-attribute task battery (MATB) (Santiago-Espada et al., 2011). NINScan supports 32 channels, with 8 channels per hemisphere in this test. Because our MEDIC fNIRS sensor only includes one source-detector pair, we further validated our findings by collecting data at two locations: the dorsolateral prefrontal cortex (dlPFC) known to exhibit changes in HbO with increasing cognitive workload, and the medial PFC, which does not exhibit changes in HbO due to cognitive workload.

Method

We recruited 23 healthy adult (21.3 ± 3.0 years; 10 males) students at Brown University. Three withdrew prior to completion. To minimize learning effects, participants completed one practice session. Within the following two sessions, participants wore one of the sensors (NINScan or MEDIC in counterbalanced order) while participants performed the task battery twice. We collected two minutes of baseline rest (eyes-closed) at the start and end of each session, and used the NASA-TLX questionnaire to assess subjective workload.

Results

N-back performance decreased with increasing difficulty level (increased response time (RT) and decreased accuracy). For the MEDIC sensor, we ran a mixed model with time dummy variables (one for each 10 second period to remove drift effect), a fixed effect of performance, and a by-subjects random effect. We found a significant effect of performance on dlPFC HbO₂ (p<.01), but not medial PFC HbO. For the NINScan, we observed increases in HbO₂ from 1- to 2- to 3-back in two channels corresponding to the border between ventrolateral prefrontal cortex (vlPFC) and dlPFC in both hemispheres (p<.05).

For MATB, we assessed behavioral performance separately for each sub-task (a tracking task, a resource management task, a communication task, and a system monitoring task). In the tracking task, performance decreased across all three measures (time within target, distance from crosshairs, and distance outside of target) as difficulty increased. For distance measures, performance improved slightly across visits. In the resource management task, performance decreased as difficulty increased for time and distance outside target. Performance improved across visits. For the communication task, hit rate decreased as difficulty increased, while RT showed an inverted u-shape with slower performance at medium difficulty. Errors were minimal, but there was an overall increase with increased difficulty. Hit rate and RT showed learning effects across visits. For the system monitoring task, hit rate decreased as difficulty increased, while RT showed an inverted u-shape with slower performance at medium difficulty. Both measures improved over the visits with unnecessary adjustments increasing over the visits, suggesting a compensatory strategy to improve task performance. Perceived effort increased with MATB difficulty for both visits, suggesting that chosen task parameters were sufficient to elicit a systematic increase in cognitive workload. Reduction of reported effort across all MATB conditions, suggests that performance was also influenced by learning effects.

When we analyzed changes in HbO₂ with difficulty level on MEDIC sensor data, we found no block-level effect of difficulty, nor correlation of HbO₂ with moment-to-moment performance. We hypothesized that participants were self-regulating cognitive workload by ignoring subtasks or cues as the difficulty increased, rather than increasing the amount of information stored in working memory in response to the increased rate of stimulus presentation. Indeed, as difficulty increased, the percentage of misses also increased.  If subjects regulate cognitive workload in this way, a combined metric, pooling information across tasks, might capture moment-to-moment changes in workload, reflecting overall workload despite transient focus on only a few subtasks. Therefore, we aggregated MATB behavioral data across subtasks. Full detail of this work is presented elsewhere (Leather et al., 2018).

We then performed exploratory analysis that revealed that regardless of difficulty, all subjects showed an increase in HbO₂ across each block. We hypothesized that the variability due to time-on-task might have hidden a relationship between behavioral performance metric and HbO₂. Therefore, we constructed a model including time-on-task. A mixed effects model to determine if the behavioral metric within each 20-second window as well as categorical regressors for time-on-task predicted the mean HbO₂ within that 20-second window (again with a by-subject random intercept). After accounting for time-on-task, we found a highly significant (p<.01) effect on HbO₂. For the NINScan, we also did not find a significant relationship between HbO₂ and task level in the MATB at the block level. We again, re-ran our analysis using our aggregated performance score, and found a significant effect at dlPFC.

Conclusions

MEDIC resulted in a commercially-available fNIRS device useful for assessing cognitive workload in real-world environments. New work is refining this sensor, improving size, comfort, ruggedness, and cost. We measured changes in HbO₂ during the n-back for which effects of cognitive workload are well-known. Results were as expected. Changes in HbO₂ were compared to a larger, better-validated sensor, the NINScan. Effects of cognitive workload on HbO₂ were similar between probes positioned over the same locations. We also used the MATB with pre-validated difficulty levels (Nelson, 2016) to examine HbO₂ changes in a task more similar to those in real-world environments. Although changes in HbO₂ examined across difficulty levels were not initially as expected for either sensor, when we aggregated performance across all sub-tasks and included time-on-task in our model, HbO₂ changes strongly correlated with difficulty level. This indicates that our new fNIRS sensor is comparable to standard sensors for detecting changes in HbO₂ that correlate with cognitive workload.

¹ Charles River Analytics
² Brown University
³ Massachusetts General Hospital
⁴ Plux Wireless Biosignals
⁵ McLean Hospital/Harvard Medical School

This work was supported by United States Army Medical Research and Materiel Command under Contract Nos. W81XWH-14-C-0018 and W81XWH-17-C-0205 and NASA Contract Nos. NNX15CJ17P and NNX16CJ08C. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Army Medical Research and Materiel Command. In the conduct of research where humans are the participants, the investigators adhered to the policies regarding the protection of human participants as prescribed by Code of Federal Regulations (CFR) Title 45, Volume 1, Part 46; Title 32, Chapter 1,  Part 219; and Title 21, Chapter 1, Part 50 (Protection of Human Participants).