Hands Free User Interfaces for Healthcare: Charles River Analytics Streamlines Capture of Medical Information with Novel Multimodal User Interfaces

Image from Charles River Analytics project

Charles River Analytics, developer of intelligent systems solutions, is building user interfaces (UIs) that streamline the capture of medical information for the US Army. The 2+ year, follow-on contract is valued at close to $1 million.

Capturing and sharing medical information across multiple providers and roles is critical to effective, coordinated and long-term care. However, it is a complex and time-consuming task that is prone to errors and especially challenging to do while providing care to the patient. With CAPTURE (Context Aware Procedure Support Tools and User Interfaces for Rapid and Effective Workflows), medical personnel will be able to input critical medical information across a range of operational environments efficiently and effectively.CAPTURE

Example of AR-based interface for medical information capture for a Tactical Combat Care Causality (TCCC) Card

CAPTURE’s state-of-the-art UI uses augmented reality for heads-up and hands-free interactions, context aware support tools, and passive semi-automated information capture. These interfaces minimize interaction costs by combining multimodal capture methods, natural language technologies, and efficient hands- and eyes-free interaction mechanisms.

CAPTURE’s augmented reality display concept will recognize key information from a dog tag and prompt the user to select the type of information for unrecognizable information (underlined text)

“We’re thrilled at the opportunity to develop truly innovative solutions that will directly affect how medical professionals capture and access critical health information while improving patient care overall,” said Stephanie Kane, Principal Investigator and Senior Scientist at Charles River Analytics. “With CAPTURE, not only can we push the state of the art in multimodal interface design, but we can also build on our growing expertise in AR and success in applying that knowledge to the medical field.”

Driven by Charles River’s SFG Toolkit product, CAPTURE combines multimodal capture methods with advanced context-aware technologies and efficient hands- and eyes-free interactions. During patient care, as providers input data, CAPTURE performs an analysis and provides real-time prompting or semi-automated data capture support. Providers can augment, correct, and flag information with hands-free interactions and context-sensitive interfaces, earmarking relevant information to enable a quicker and easier post-care review when the provider has more time for dedicated interaction.

In August, Kane presented CAPTURE at the Military Health System Research Symposium.

CAPTURE is one of Charles River’s Healthcare Support and Training efforts, which addresses medical skills training, therapy and decision support tools, as well as sensor and sensing technologies. We also developed the VITAMMINS medical simulation and tutoring system, and the STAT tablet-based training system, which presents a virtual patient in multiple trauma scenarios for efficient and effective learning, rehearsal, and assessment.


This work is supported by the US Army Medical Research and Materiel Command under Contract W81XWH-18-C-0099. The views, opinions and/or findings contained in this report are those of the author(s) and should not be construed as an official Department of the Army position, policy or decision unless so designated by other documentation.

Solutions to serve the warfighter, technology to serve the world®

Charles River Analytics brings foundational research to life, creating human-centered intelligent systems at the edge of what’s possible, through deep partnerships with our customers. 

To learn more about Charles River or our current projects and capabilities, contact us