“Our approach to managing UxVs is in stark contrast with how robot operators currently direct unmanned platforms—they use cumbersome, hands-on, head-down controllers. Our goal with SHEPARD is to develop controls that let robots seamlessly integrate into human teams.” – Camille Monnier, SHEPARD Principal Investigator
Unmanned vehicles (UxVs) are vital to military operations, although their widespread integration faces significant challenges. Current control schemes require a trained operator to remotely control an unmanned system to execute a Commander’s orders. This increases the complexity of the command structure, introduing latency and reducing situational awareness.
Developing an Intuitive Interface for the US Army
In response to the US Army’s need for advanced, intuitive interfaces, we designed and prototyped a system for Supervisory HMI Enabling Practical Autonomous Robot Direction (SHEPARD). Our human-machine interface (HMI) consists of a portable software package and a modular hardware kit, including a camera and a lightweight, wearable operator control unit.
SHEPARD uses smart devices—like a watch—for easy communication with military robots. We developed a context-sensitive control module that enables natural, reliable control of multiple platforms by fusing multi-modal inputs, such as voice and gestures, with operational context.
Effortless Human-Robot Communication
By removing the need for a Soldier to teleoperate an unmanned system, SHEPARD allows the robotic platform to become a true support agent. SHEPARD’s hands-free HMI includes a context-sensitive control module based on a model of operator intent. By fusing context cues with inputs provided from multiple control modalities (e.g., voice, gesture), the system infers a model of operator intent that can be used to resolve ambiguous commands and provide situation-appropriate feedback. SHEPARD supports several gesture control configurations to make human-machine communication more natural than ever.
Clearpath Husky platform responding to gesture and voice commands
Related Work
MANTA – a platform-independent, natural control and autonomy robot appliqué that enables a user to easily direct one or more host platforms to perform a range of autonomous behaviors.
