Charles River Analytics, a GRVTY company, is creating a research platform for scientists and engineers to evaluate user-interface design concepts within human‑AI teaming environments. Following a successful initial effort, the Air Force Research Laboratory (AFRL) 711th Human Effectiveness Directorate awarded an additional $1.7 million contract to further refine and develop a testbed. As AI technology continues to become more prevalent and complex, researchers need better ways to evaluate how AI and humans can safely and effectively collaborate.
“When you put a human and AI together, the hope is that performance is better than either could accomplish alone,” says Dr. Nicolette McGeorge, Senior Scientist at Charles River Analytics and Principal Investigator on the project. She adds, “But there can be catastrophic consequences if you don’t effectively design for the human to use the technology collaboratively with the AI capability.”
The Charles River team is developing Human Autonomy Teaming Testbed for Research (HATTER), a low‑code, intuitive platform that enables researchers to test human‑AI interactions without extensive software engineering. The platform will be a medium-fidelity research tool that provides more detail than low-fidelity tools while offering greater flexibility and ease of use than high-fidelity tools. The user-friendly testbed aims to help researchers directly measure and quantitatively assess human‑AI performance within candidate human‑AI systems.
The team’s approach is grounded in human factors techniques and cognitive systems engineering, an interdisciplinary approach to analyzing, modeling, and designing engineered systems. They are expanding on proven research methods initially developed by their collaborators at The Ohio State University’s Cognitive Systems Engineering Laboratory (CSEL). The testbed features a component library that researchers can use to easily compose a candidate user interface, design experiments, and simulate AI systems’ performance in various operationally relevant scenarios involving human interactions.
“If we start early with designing these human‑AI systems for better interactions, we can decrease risk and achieve overall better human‑AI support,” says McGeorge. She adds, “We find it less costly and less complex to do it early in the design and development cycle.”
While initially developed to meet AFRL’s specific needs for a medium-fidelity framework for evaluating design concepts with respect to human‑AI teaming, HATTER is flexible enough for broader research and can be used for additional domains beyond military applications. It supports interface concept evaluation through quantitative performance measurement while enabling easier, more flexible human‑AI research. In this way, the project addresses the challenge of effectively combining human expertise with AI capabilities within emerging AI‑infused systems.
Contact us to learn more about HATTER and our other human‑AI teaming capabilities.
This material is based upon work supported by the US Air Force under Contract No. FA2384-25-C-B027. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the US Air Force.
