CRISIS
A system using heterogeneous crowds for complex analysis problems
Crowdsourcing using Intelligent Supervision to address Information Requirements in Crisis Situations
Marine Expeditionary Units (MEUs) are often first responders for such crises as natural disasters and regional instability. To respond effectively, MEUs must quickly and accurately analyze large amounts of information collected on the ground. Fortunately, crowdsourcing offers opportunities to efficiently leverage vast, knowledgeable, and unexploited resources in the form of local and military populations that can support information requirements (IR) and analysis needs. However, Marines need tools to access and manage these distributed human processing resources.
The Charles River Analytics Solution
Scientists and software engineers developed a framework for Crowdsourcing using Intelligent Supervision to address Information Requirements in Crisis Situations, or CRISIS. The CRISIS framework employs diversely skilled crowds to address sensing and analysis problems for intelligence analysts. CRISIS incorporates a unique provenance-modeling approach to model crowd performance with limited ground truth information. The provenance model learns which users or user profiles to trust most for specific tasks.
The Benefit
CRISIS provides a proprietary provenance modeling approach to analyze the skill and trustworthiness of contributors while maintaining the anonymity of crowd workers and requiring limited ground truth comparison to learn about the crowd. Using simulated crowds, CRISIS has been able to identify deceptive and biased crowd members with high accuracy, and has produced significant improvements over baseline MEU data-gathering scenarios. Using live and synthetic tests, Charles River has demonstrated an initial CRISIS capability for crowdsensing (using a crowd to gather data about the crowd’s surrounding environment) that reduces error when compared to an approach based on manual sampling. This capability could provide critical information from across an operational environment in a crisis situation in less time than an equally-accurate manual sampling of random locations within that environment.
As with the MEU scenarios, CRISIS has great applicability to non-governmental organizations challenged with crisis-response tasks. In addition to supporting these rapid responses to emerging crisis situations, the CRISIS technology can be used to improve commercial crowdsourcing outcomes and the performance of existing crowdsourcing platforms. Additionally, this technology can be used by the media community to discover details of a developing situation and human responses to challenge questions, resulting in more accurate and complete news media stories.
Contact us to learn more about CRISIS and our other capabilities.
This material is based upon work supported by the Office of Naval Research under Contract No. N00014-15-C-0151. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Office of Naval Research.
