Unmanned Systems Technology XIV, Proceedings of SPIE, vol 8387, Baltimore, MD (April 2012).

Current generation unmanned ground vehicle (UGV) control systems typically require operators to physically control a platform through teleoperation, even for simple tasks such as travelling from one location to another. While vision-based control technologies promise to significantly reduce the burden on UGV operators, most schemes rely on specialized sensing hardware, such as LIDAR or stereo cameras, or require additional operator-worn equipment or markers to differentiate the leader from nearby pedestrians. We present a system for robust leader-follower control of small UGVs using only a single monocular camera, which is ubiquitous on mobile platforms. The system allows a user to control a mobile robot by leading the way and issuing commands through arm/hand gestures, and differentiates between the leader and nearby pedestrians. The software achieves this by integrating efficient algorithms for pedestrian detection, online appearance learning, and kinematic tracking with a lightweight technique for camera-based gesture recognition.