Robert Hyland, Kenneth R. Lu, Spencer Lynn, Ph.D, Stephen J. Marotta, James Niehaus, Ph.D., William Norsworthy Jr., Avi Pfeffer, Ph.D., Curtis Q. Wu, Bryan Loyall, Ph.D.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, Florida (29 November 2023)
How can we build artificial intelligence (AI) that robustly recognizes how well a team is doing from behavioral data that exhibits the full range of human complexity and dynamics? One method is cognitive inversion. An AI with a causal model of human behavior that is sufficiently dynamic to account for behavioral variability and teammate interactivity and scoped to a set of tasks and interactions of interest, combined with a probabilistic program inference, can invert that behavioral model to generate hypotheses about the underlying goals and causes of observed behavior. As a tutor, coach, or teammate, the AI can then intervene to assist when needed. Here, we describe our prototype cognitive inversion system called Prescient, Socially Intelligent Coach (PSI-Coach) and its supporting components. PSI-Coach monitors team members to recognize their goals, mental states, and behaviors from dynamic streams of actions by combining probabilistic programming inference with a cognitive architecture designed to capture human variation. PSI-Coach uses those recognized cognitive states to infer a team’s shared mental models and whether they are in alignment or skewed; analyze these goals, mental states, behaviors, and shared mental models to compute practical, real-time team performance indicators; and integrate all of this information with interactive-narrative technology to plan minimally intrusive, effective, strategically timed interventions that help to improve team performance. In experiments, we demonstrated the ability of cognitive inversion to automatically identify team process problems unique to different teams and their situation dynamics, and, based on those results, we show PSI-Coach’s ability to provide timely, tailored intervention content that improved team processes. Cognitive inversion showed a 35% increase over rule-based comparison systems’ real-time inferences (p<0.05), and led PSI-Coach to exhibit a 42%–68% increase in agreement with human coaches on interventions over a baseline inference method (p<0.05).
For More Information
To learn more or request a copy of a paper, contact Rob Hyland.
(Please include your name, address, organization, and the paper reference. Requests without this information will not be honored.)
