SHERLOCK
A platform for prototyping solutions on physiological, neurological, and behavioral state
The Situation
Across many careers, people face alternating periods of high and low attention and cognitive workload, which can result in impaired cognitive functioning and can be detrimental to job performance. For example, some professionals (e.g., firefighters, emergency medical personnel, doctors and nurses working in an emergency room, pilots) experience long periods of low workload (boredom), followed by sudden, high-tempo operations where they may need to respond to an emergency and perform at peak levels. Conversely, other professionals (e.g., air traffic controllers, financial investors, analysts in many fields) experience long periods of high workload and multitasking, and are constantly on the verge of a sudden disruption that could lead to mistakes. An unobtrusive system to measure, assess, and predict cognitive workload could warn individuals, their teammates, or their supervisors when steps should be taken to augment cognitive readiness.
The Charles River Analytics Solution
Scientists and software engineers at Charles River Analytics developed Sherlock™ as an open and extensible software and hardware platform that enables users to rapidly prototype applications that collect, analyze, visualize, and reason about human physiological, neurological, and behavioral states. Our Sherlock platform enables rapid integration of custom or commercially available off-the-shelf (COTS) hardware components (e.g., sensors, data transmitters, data servers) using an open data exchange layer that requires minimal development to meet operational needs.
Specific capabilities within the Sherlock platform include the following:
- Enables synchronized physiological, neurological, and behavioral signal acquisition for a range of human state variables
- Plugin-based architecture minimizes configuration times such that each plugin only needs to be written once and then can be used with any application built on top of the Sherlock platform
- Hardware plugins provides native support for numerous COTS sensor peripherals and the ability to readily integrate custom sensors
- Analytic plugins enable transformation and analytic operations to be carried out on data acquired through any active hardware plugin (i.e., any heart rate sensor will work with a single heart rate analytic plugin)
- Integrated analytic plugins provide out of box support for:
- Real-time monitoring of performance-relevant constructs (e.g., cognitive workload, stress, injury status)
- State classification based on a proven ensemble approach including:
- Basic statistical methods (e.g., weighted averaging, Markov chain Monte Carlo techniques for sampling data sets)
- Expert-driven and data-driven probabilistic models (e.g., Bayesian networks, causal influence models)
- Deep learning architectures and techniques (e.g., convolutional neural networks)
- Capabilities validated through evaluations in real-world environments (e.g., high-fidelity training simulations, battlefield casualty care training events) and controlled empirical studies
- Intuitive graphical user interface presents streaming and acquired data and system outputs for all users
Sherlock has powered many of our efforts, including the CAPT PICARD system for NASA
The Benefit
Sherlock includes components for data transmission and storage, including a multicast message queue that supports software components written in multiple languages and on multiple platforms, and a HIPAA-compliant database server. Sherlock solutions have been deployed to mobile devices, to web browsers, and as client-side applications for personal computers. The platform also supports rapid prototyping of user interface components so users can intuitively create custom data processing models and present acquired and processed data in a format that fosters end-user trust and understanding. The Sherlock platform provides software solutions for advanced modeling and analytic techniques built on a solid foundation of Charles River Analytics products, such as AgentWorks™ for model composition, BNet® for probabilistic reasoning, and Connect™ for network analysis and visualization. These products have been developed and hardened for deployment in systems that have transitioned to government and commercial products, and are in use in a range of projects for multiple customers.
Sherlock has been successfully incorporated in several efforts at Charles River, including:
- ADAPTER – A toolkit that models individual and team cognitive state and recommends performance optimization strategies
- MEDIC – A system to automatically sense indicators of cognitive workload
- CAPT PICARD – A system to measure, assess, and predict cognitive workload to assist astronauts or mission control in responding more effectively
- SOMBA – A standoff method for measuring physiological signals related to health and psychological state
- ADVISOR – A system to assess and diagnose vestibular indicators of soldiers operational readiness
