73^rd Department of Defense Human Factors Engineering Technical Advisory Group Meeting (DoD HFE TAG), Aberdeen, MD (April 2019)

Among the multitude of training techniques the DoD employs, virtual simulation continues to grow due to its ability to more closely emulate real-world contexts. However, these high-fidelity virtual simulations also often require prohibitively large amounts of time and money to develop, maintain, and integrate.

Under VIRTUOSO, a 2.5-year ARL-funded effort, Charles River Analytics is developing the VIRTUOSO Software Development Kit (VSDK), a Unity-based XR SDK planned to become open-source in 2019, designed for developers to rapidly generate immersive, consistent XR-based simulations. VSDK provides a hardware-agnostic XR development architecture and a suite of XR development tools and libraries to enable rapid prototyping of immersive XR experiences and advanced XR interactions, with a focus on supporting natural human interactions and dynamic environments to foster more realistic simulations and training. To enable naturalistic control in virtual environments, we are building support for a variety of XR peripherals (e.g., data gloves, motion capture suits, haptic wearables) which can be used to enhance simulation immersion (e.g., by supporting gesture-based controls, interaction with physical tracked objects, and environmental haptic feedback).

Additionally, we analyzed common user interactions and procedures specific to the training domain (e.g. combat medic training under our VIRTUOSO effort), to develop a requisite library of gestures, interaction areas, and interactable objects. We then integrate this naturalistic interaction support with dynamic simulated environments, including intelligent agents and state machines, speech control libraries for voice-based interactions with non-player characters (NPCs), and time- and event- based scripting, to better emulate the dynamic nature of the real world.