A Novel Approach to Automated Assessment Generation Using Semantic Extraction
Terry Patten1, Rachel Amey2, Joanne Barnieu3, Clarence Dillon3, Jennifer Harvey3, Sean Shiverick3, Michael Smith3, Steve Hookway1 Paper Session, MODSIM 2023, Norfolk, VA (May 2023). 1 Charles River Analytics 2 U.S. Army Research Institute for the Behavioral and Social Sciences 3 ICF Rapid and reliable individual-level assessments are critical to developing effective and capable Army Soldiers and meeting the […]
CrowdSim: A Generative Model of Crowdsourced Survey Responses
Michael Lepori, Derek Thayer, Sean Guarino, Leonard Eusebi Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, Florida (1 December 2021) Career development relies on an understanding of possible future roles, available training experiences, and current skills. To target training where it will be most effective, trainees and instructors must understand how these elements interact, which […]
Cyber Reactive Adversary Framework for Training
Sean Guarino1, William Norsworthy1, David Kelle1, John Steigerwald1, Timothy Ho1, Dorsey Wilkin2 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, Florida (29 November 2023) Networks have become a critical background for military operations as adversaries and hackers become increasingly prolific and proficient at cyber warfare. Despite this, cyber training has remained focused on large-scale exercise […]
Immersive Space Operations Training in Extended Reality
Daniel Stouch, Sean Guarino, Dan Duggan, Susan Latiff, Robert Hyland, Kimberly Brady Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, Florida (29 November 2023) The training of next-generation space operators for both commercial and military capabilities relies largely on analog physical models and PowerPoint lectures. Consequently, new operators do not often fully grasp the fundamentals […]
AI Inference of Team Effectiveness for Training and Operations
Robert Hyland, Kenneth R. Lu, Spencer Lynn, Ph.D, Stephen J. Marotta, James Niehaus, Ph.D., William Norsworthy Jr., Avi Pfeffer, Ph.D., Curtis Q. Wu, Bryan Loyall, Ph.D. Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, Florida (29 November 2023) How can we build artificial intelligence (AI) that robustly recognizes how well a team is doing from […]
Applying Adaptive Intelligent Tutoring Techniques to Physical Fitness Training Programs
Voge, J.1, Negri, A.1, Woodall, P.1, Thayer, D.1, Ruby, B.2, Hailes, W.2, Reinert, A.3, Niehaus, J.1, and Lynn, S.1 To appear in Proceedings of the 3rd International Conference on Adaptive Instructional Systems, held as part of the 23rd International Conference on Human-Computer Interaction, Washington, DC (July 2021) Adaptive Training Protocols (ATP) is a collection of […]
Designing Serious Games to Train Medical Team Skills
McDermott, A., Niehaus, J., and Weyhrauch, P. Virtual Interservice/Industry Training, Simulation, and Education Conference (vI/ITSEC) (November 2020) More access to effective medical team training across the DoD can significantly improve patient outcomes and save lives (Hughes et al., 2016). The DoD and medical communities need implementations of team training protocols that do not require extensive time […]
Performance Assessment in a Virtual Simulation for Integrated Austere Medical Operations Training
Bauchwitz, B.1, Makivic, M.1, Manning, W.1, Niehaus, J.1, Broach, J.2, Lancette, P.2,3, Garrison, C.4, Ritter, F.4, and Weyhrauch, P.1 Virtual Interservice/Industry Training, Simulation, and Education Conference (vI/ITSEC) (November 2020) The DoD is preparing for future austere environments where it will not be possible to sustain the current level of medical operations across the chain of […]
The Effect of Task Fidelity on Learning Curves
Ritter, F.1, McDermott, A.2 MathPsych/International Conference on Cognitive Modeling 2020 What is the effect of level of simulation fidelity on learning and then on performance in the target task? We consider an example of an electronic maintenance training system with two levels of fidelity: a high fidelity (HiFi) simulation that basically takes as much time […]
A Pilot Study Comparing Two Naturalistic Gesture-Based Interaction Interfaces to Support VR-Based Public Health Laboratory Training
Voge, J., Flowers, B., Herrera, N., Duggan D., and Wollocko, A. 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Atlanta, GA (March 2020) Training opportunities for public health laboratories (PHLs) are limited, and traditional wet laboratory training is expensive and potentially hazardous. Virtual reality presents a promising complement to existing training and provides the […]
Assessing Trainee Readiness for Highly Demanding Jobs
Bracken, B. Industrial Management (March/April 2020) An ongoing challenge facing trainers, trainees and future employers is determining when a student is ready to move on and take on responsibilities in a profession where performance really matters. To bring this insight into assessments of trainee readiness for the job, a new technology has been developed to […]
A Human Factors Approach to Networked Virtual Reality and Augmented Reality Interactions with Physiologically-Driven Believable Agents
Kingsley, C. and Jenkins, M. 2019 Applied Human Factors and Ergonomics (AHFE) International Conference, Washington, DC (July 2019) Charles River Analytics has been leading numerous research and development efforts within the Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) domains, collectively Extended Reality (XR), including an exploration of the human factors considerations for […]
An Extended Reality (XR) SDK to Foster Development of Realistic XR Simulations Through Naturalistic Control and Dynamic Virtual Environments
Kingsley, C., Duggan, D., and Jenkins, M. 73rd Department of Defense Human Factors Engineering Technical Advisory Group Meeting (DoD HFE TAG), Aberdeen, MD (April 2019) Among the multitude of training techniques the DoD employs, virtual simulation continues to grow due to its ability to more closely emulate real-world contexts. However, these high-fidelity virtual simulations also often […]
A Toolkit to Help Integrate Humans with Virtual Environments, Intelligent Simulations (FSMs), and Artificial Cognitive Systems (NPCs)
Kingsley, C. and Jenkins, M. Proceedings of the 2019 Intelligent Human Systems Integration (IHSI) International Conference, San Diego, CA (February 2019) Charles River Analytics is developing an open-source, Unity-based Extended Reality (XR) SDK for development of immersive simulations, currently focused on medical simulations. Our work-to-date has focused on integrating humans into virtual environments through enhanced […]
Use of a Portable Functional Near-Infrared Spectroscopy (fNIRS) System to Examine Team Experience During Crisis Event Management in Clinical Simulations
Xu, J.1, Slagle, J.2, Banerjee, A.3, Bracken, B.4, and Weinger, M.5 Frontiers in Human Neuroscience (05 March 2019) Objective: The aim of this study was to investigate the utilization of a portable functional near-infrared spectroscopy (fNIRS) system, the fNIRS PioneerTM, to examine team experience in high-fidelity simulation-based crisis event management (CEM) training for anesthesiologists in operating rooms. Background: Effective […]
Exploring Extended Reality as a Simulation Training Tool Through Naturalistic Interactions and Enhanced Immersion
Duggan, D.1, Kingsley, C.1, Mazzeo, M.2, and Jenkins, M.1 International Conference on Human-Computer Interaction: Virtual, Augmented and Mixed Reality Applications and Case Studies, Orlando, FL (June 2019) Augmented reality (AR), mixed reality (MR), and virtual reality (VR) technologies – collectively extended reality (XR) – have been discussed in the context of training applications since their […]
Characterizing the Limits of Visual Field Augmentation in Augmented Reality: Psychophysical Foundations for Design and Mode Transition Acclimation Strategies
Jenkins, M., Kingsley, C., and Flowers, B.A. Panel Presentation at the 22nd Annual Applied Ergonomics Conference, New Orleans, LA (March 2019) The impact of augmented reality displays on the human perceptual system will vary across headsets. Because of this, the display used for any solution should be selected with an awareness of the interaction between the […]
Modeling Framework Used to Analyze and Describe Junctional Tourniquet Skills
Bauchwitz, B.1, Curley, T.1, Kwan, C.2, Niehaus, J.1, Pugh., C.2, and Weyhrauch, P.1 Military Medicine, Volume 184, Issue Supplement 1, Pages 347–360 (March-April 2019) Medical educators have acknowledged the importance of simulation training in developing procedural skills. While simulation training in other disciplines has benefitted from evaluations of users’ skill acquisition, the majority of medical training […]
A Computer-Based Tutor to Teach Nursing Trauma Care that Works as an Adjunct to High Fidelity Simulation
Garrison, C.1, Ritter, F. E.2, Bauchwitz, B.3, Neihaus, J.3, and Weyhrauch, P.3 Presented at the 19th Annual International Meeting on Simulation in Healthcare, San Antonio, TX (January 2019) Introduction: Educators are challenged to prepare nurses to care for low volume, high acuity clinical problems such as trauma. It is difficult to provide learners with hands-on practice without compromising the quality of […]
Strengthening Health and Improving Emotional Defenses (SHIELD)
Elkin-Frankston, S., Wollocko, A., and Niehaus, J. In: Schmorrow D., Fidopiastis C. (eds) Augmented Cognition: Users and Contexts. AC 2018. Lecture Notes in Computer Science, vol 10916. Springer, Cham (03 June 2018) Relaxation techniques such as deep breathing, and meditation can be used to gain more control of how individuals respond to stressful situations. While these techniques are becoming […]
Using Multi-Sensor Voting for Resilience in VR Biofeedback Games
Flowers, B1., Tatoian, R2., Witkowski, A2., Katenka, N2., and Herve2, J. 2018 IEEE Games, Entertainment, Media Conference (GEM) (August 2018). Biofeedback games are often developed to teach self-management skills to children. Unfortunately, the underlying system for many of these games are designed for laboratory environments and are not built to accommodate sensor movement and noise […]
Development of Virtual Simulations for Medical Team Training: An Evaluation of Key Features
Howe, J.1, Puthumana, J.1, Hoffman, D.1, Kowalski, R.1, Weldon, D.1, Miller, K.1, Weyhrauch, P.2, Niehaus, J.2, Bauchwitz2, B., McDermott, A.2, and Ratwani, R.1 Poster presentation for the International Symposium on Human Factors and Ergonomics in Health Care, Boston, MA (March 2018) Teamwork is an important factor in reducing medical errors and increasing clinical efficiency. Medical […]
Enabling Military Virtual Reality and Mixed Reality Simulations with a Flexible Peripheral Framework and Automated Assessments
Jenkins, M., Irvin, S., Wong-Faull, D., and Bird, L. Presented at the International Conference on Applied Human Factors and Ergonomics (AHFE), Orlando, FL (July 2018). The military virtual training and simulation market is forecast to be a $6B+ market through 2020, driven by the increasing immersiveness of virtual reality (VR), augmented reality (AR), and mixed […]
Reduced Interference in Working Memory Following Mindfulness Training is Associated with Increases in Hippocampal Volume
Greenberg, J., Romero, V., Elkin-Frankston, S., Bezdek, M., Schumacher, E., and Lazar, S. In Brain Imaging and Behavior, Springer US, 1931-7565 (2018). Proactive interference occurs when previously relevant information interferes with retaining newer material. Overcoming proactive interference has been linked to the hippocampus and deemed critical for cognitive functioning. However, little is known about whether and how this ability […]
Modeling Training Efficiency and Return on Investment for Adaptive Training: GIFT Integration
Generalized Intelligent Framework for Tutoring (GIFT) (May 2018) Adaptive training promises more effective training by tailoring content to each individual. Where non-adaptive training may be just right for one segment of the student population, there will be some students that find it too easy while others find it too difficult. Another, often ignored benefit of […]
No Evidence of Cognitive Improvement following tDCS-enhanced Mindfulness Meditation or Game-Based Brain Training
Elkin-Frankston, S.1, Lieberman, G.2,3,4,5, Guarino, S.1, Eusebi, L.1, Bezdek, M.6, Hunter, M.2,3, Lazar, S.7, Witkiewitz, K.2, Clark, V.2,3, Schumacher, E.6, and Romero, V.1 The 4th Annual BRAIN Initiatve Investigators Meeting, Rockville, MD (April 2018). Measures of fluid intelligence are highly predictive of performance across a wide range of contexts, including academic performance and career success, […]
Modeling and Comparing Nurse and Physician Trauma Assessment Skills
Bauchwitz, B., Niehaus, J., and Weyhrauch, P. Military Medicine, Volume 183, Issue suppl_1, 1 March 2018, Pages 47–54 (April 2018). Medical providers must master a large number of complicated tasks to deliver quality care and minimize unwanted clinical outcomes. In order to optimally train these tasks, medical training systems would benefit from models of skill […]
Development of Screen-based Simulation Content for Training and Assessing Trauma Nursing Proficiency
Bauchwitz, B.1, Jastrzembski, T.2, Millwater, T.2, Gunzelmann, G.2, Niehaus, J.1, and Weyhrauch, P.1 Presented at the Military Health System Research Symposium (MHSRS), Kissimmee, FL (August 2018) Background: Trauma care is crucial to the DoD, but current training may be non-optimal. Military medical providers typically receive classroom instruction in courses such as the Trauma Nurse Core Course […]
Comparison of a Custom Functional Near-infrared Spectroscopy Sensor, a Peripheral SpO2 Sensor, and a Standard Laboratory Sensor (Biopac) for RR-Interval Assessment
Bracken, B.1, Amazeen, G.2, Likens, A.2, Demir, M.2, and Gibbons, C.2 Presented at the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018), Funchal, Portugal (January 2018) Across many careers, individuals face alternating periods of high and low cognitive workload which can impair cognitive function and undermine job performance. We have designed and […]
Modeling Junctional Tourniquet Skills from Empirical Data
Bauchwitz, B.1, Kwan, C.2, Curley, T.3 Niehaus, J.1, Pugh, C.2, and Weyhrauch, P.1 Presented at the Military Health System Research Symposium (MHSRS), Kissimmee FL (August 2017) Background The control of junctional hemorrhage, a serious life-threatening issue facing combat personnel, has benefited from recent technological advances such as the development of the SAM® Junctional Tourniquet (SJT) and Abdominal […]
A System to Monitor, Extract, and Decode Indicators of Cognitive Workload (MEDIC) in Realistic, High-motion Environments
Bracken, B.1, Elkin-Frankston, S.1, Palmon, N.1, deB Frederick, B.2 Presented at the Military Health System Research Symposium, Kissimmee, FL (August 2017). Background Army medic training often includes high-fidelity simulations. Trainers currently infer competence by observation alone—a challenging task. We designed a system to continuously monitor physiological proxies for cognitive workload to augment performance observations. Assessing cognitive […]
A System to Monitor Cognitive Workload in Naturalistic, High-motion Environments
Bracken, B.1, Elkin-Frankston, S.1, Palmon, N1, Farry, M.1, and deB Frederick, B.2 Presented at the 1st International Symposium on Human Mental Workload: Models and Applications (H-WORKLOAD 2017), Dublin, Ireland (June 2017) Across many careers, individuals face alternating periods of high and low attention and cognitive workload can impair cognitive function and undermine job performance. We have […]
Strengthening Health and Improving Emotional Defenses (SHIELD)
Elkin-Frankston, S., Niehaus, J., and Wollocko, A. Presented at the Second Annual Global Brain Health and Performance Summit, Columbus, Ohio (April 2017) The psychological stress experienced by Marines has a profound impact on health and performance that reaches beyond the individual; it affects job performance, interactions with family and peers. The DoD has devoted substantial resources […]
Virtual Patient Simulation with Objective Metrics for Primary and Secondary Trauma Assessment
Weyhrauch, P.1, Niehaus, J.1, Bauchwitz, B.1, Broach, J.2, Lancette, P.J.2,3, and Ritter, F.E.4 Presented at the Military Health System Research Symposium (MHSRS), Kissimmee FL (August 2017) Background One commonly used approach to training trauma assessment and other emergency medicine skills is simulation-based training, allowing trainees to practice their skills with simulated patient scenarios. This approach often […]
Objective Sensor-based Feedback Improves Performance of Skill in Applying Junctional Hemorrhage Control Devices
Bauchwitz, B.1, Kwan, C.2, Niehaus, J.1, Pugh, C.2, and Weyhrauch, P.1 Presented at the Military Health System Research Symposium (MHSRS), Kissimmee FL (August 2017) Background Junctional and inguinal bleeding from battlefield injuries is a significant and challenging problem. Inventors have developed new devices, such as the SAM® Junctional Tourniquet (SJT) to address these injuries. While several […]
No Evidence of Improved Intelligence with Mindfulness Meditation or Brain-Training Combined with tDCS
Elkin-Frankston, S., Guarino, S., Romero, V., Lieberman, G., Hunter, M., Jones, A., Trumbo, M., Coffman, B., McCallion, E., Armenta, M., O’Sickey, A., Brown, D., Roos, C., Robinson, C., Combs, A., Witkiewitz, K., Clark, V., Schumacher, E., Bezdek, M.; Martin, N., Kumar, A., and Kaimal, A. Poster presented at the Cognitive Neuroscience 23rd Annual Meeting, New York, NY […]
Enhancing Hazard Awareness with a Mobile Application for High Risk, High Consequence Avalanche Terrain Decisions
Jenkins, M., Gale, K., and Young, D. Presented at the 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), San Diego, CA (March 2016) Design of decision-support tools for high risk, high consequence decision making has been an area of focus for many high profile domains (e.g., aviation, military […]
Enhancing Hazard Awareness with a Mobile Application for High Risk, High Consequence Avalanche Terrain Decisions
Gale1, K., and Jenkins2, M. Presented at the International Snow Science Workshop (ISSW), Breckenridge, CO (October 2016) The avalanche industry understands that to save lives, we need to understand human behavior. The scientific discipline most directly affiliated with human behavior is behavioral science. The avalanche industry has a unique opportunity to collaborate with behavioral science leaders […]
Second Validation of a System to Monitor, Extract, and Decode Indicators of Cognitive Workload (MEDIC)
Bracken, B.1, Negri, A.1, Amazeen, P.3, Likens, A.3, Fedele, M.4, Demir, M.4, Palmon, N.1, deB Frederick, B.2, and Cooke, N.4 Presented at the Medical Health Systems Research Symposium (MHSRS), Kissimmee, FL (August 2016) Background: Military medical personnel are deployed to operational environments where their success saving lives depends on their ability to act quickly and effectively, both […]
Analysis and Comparison of Nurse and Physician Team Trauma Assessment Skills
Bauchwitz, B., Niehaus, J., and Weyhrauch, P. Presented poster at the Medical Health Systems Research Symposium (MHSRS), Kissimmee, FL (August 2016) Background: Trauma assessment is key to patient outcomes. One approach to training trauma assessment is simulation-based training using patient scenarios to allow trainees to practice skills. Training can be individual or team-based, depending on the […]
Monitoring, Extracting, and Decoding Indicators of Cognitive Workload (MEDIC)
Bracken, B.1, Palmon, N.1, Frederick, B.2, Cooke, N.3, Romero, V.1, Koelle, D.1, and Pfautz, J.1 Presented at the 2015 Military Health System Research symposium, Ft. Lauderdale, FL (August 2015). Background: US military medical personnel may be deployed to a variety of operational environments where their success saving lives depends on their ability to act quickly and effectively, […]
Monitoring, Extracting, and Decoding Indicators of Cognitive Workload (MEDIC)
Bracken, B.1, Palmon, N.1, Frederick, B.2, Cooke, N.3, Romero, V.1, Koelle, D.1, and Pfautz, J.1 Poster to be presented at Neuroscience 2015, Chicago, IL (October 2015) Background: The success of emergency medical personnel in saving lives depends on their acting quickly and effectively, as individuals and as teams. Training must go beyond individual skills to include team […]
A Program for Strengthening Health and Improving Emotional Defenses (SHIELD)
Elkin-Frankston, S., Romero, V., and Wollocko, A. To be presented at the 2015 Defense Energy Innovation Summit, Austin, TX (December 2015) The psychological stress experienced by Marines has a profound impact on health and performance that reaches beyond the individual; it affects job performance, interactions with family and peers, and may lead to increased prevalence of […]
Developing Authoring Tools for Skills Models that Enable Adaptive Game-Based Maintenance Training
Guarino, S., Weyhrauch, P. and Niehaus, J. Presented at the Interservice/Industry Training, Simulation & Education Conferece (I/ITSEC), Las Vegas, NV (December 2015) While game-based maintenance training provides a powerful, personalized approach to address individual training needs, it can be costly to update immersive game engines to address new training objectives. Challenges lie not only in the […]
CREATEing a Semantic Repository for Instructional Material
Hookway1, S., Metzger1, M., Sliva1, A., Patten1, T., Lasser1, S., Wallace1, J., and Long2, R., International Semantic Web Conference (ISWC) 2015 Workshop on LINKed EDucation, LINKED 2015, Bethlehem, PA (October 2015) This document describes the prototype Collaborative Repository and Exchange for Adaptable Training Environments (CREATE) framework. The CREATE prototype was developed to demonstrate how the […]
Changes in Cortical Thickness after tDCS-Enhanced Brain Training and Mindfulness Practice
Lieberman1, G., Hunter1, M., Bezdek2, M., Trumbo1, M., Jones1, A., Coffman3, B., McCallion1, E., Armenta1, M., O’Sickey1, A., Brown1, D., Roos1, C., Robinson1, S., Martin2, N., Schumacher2, E., Guarino4, S., Eusebi4, L., Elkin-Frankston4, S., Romero4, V., Witkiewitz1, K., and Clark1, V. Presented at the Organization for Human Brain Mapping (OHBM) meeting, Honolulu, Hawaii (June 2015). The aim […]
How to CREATE Domain Content for Intelligent Tutoring Systems
Patten1, T., Metzger1, M., Hookway1, S., Sliva1, A., Lasser1, S., Wallace1, J., and Long2, R. Generalized Intelligent Framework for Tutoring (GIFT) Users Symposium (GIFTSym3), Orlando, FL (June 2015) A critical—but often overlooked—step in authoring intelligent tutoring systems is the initial retrieval and extraction of domain content. This step is increasingly challenging as the number and size of […]
Designing Instructor Support Tools for Virtual Shiphandling Training
Voshell, M., Kilgore, R., Hogan, C., and Young, D. Presented at the Interservice/Industry Training, Simulation & Education Conferece (I/ITSEC), Las Vegas, NV (December 2015) Practice and experience are fundamental to seamanship and shiphandling training, but are significantly limited by platform availability and at sea time. With hands-on experience and at-sea assignments available only to a subset […]
Translating Clinical Vestibular Function Assessment Procedures to a Portable, Low-Cost, Virtual Reality Solution
Wollocko, A., Jenkins, M., Irvin, S., and Farry, M. To be presented at the 2nd International Conference on Knowledge and Software Engineering (ICKSE 2015), Rome, Italy (December 2015) Assessment of vestibular function in adult populations is typically conducted in controlled clinical environments by trained otolaryngologists using costly, specialized medical screening equipment. This limits the ability of […]
Microgames for Training Perceptual Skills
Guarino, S.1, Jarvis, R.1, Mahoney, S.1, and Connell, M.2 Presented at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), Orlando, FL (December 2014). A common challenge in military training lies in motivating personnel to practice and retain lessons in the limited time they have available. A training methodology that can exploit the moments of “micro-boredom” that […]
Supporting Procedural and Perceptual Learning in Laparoscopic Surgery
Lou, Y.1, Flinn, J.1, Ganapathy1, S., Weyhrauch, P.2, Niehaus, J.2, Myers, B.1, and Cao, C.1 To be presented at the 58th Annual Meeting of the Human Factors and Ergonomics Society, Chicago, IL (October 2014) Expertise in surgical performance requires mastery of both technical skills such as suturing, and nontechnical skills such as perceptual and procedural knowledge. […]
