Eduardo Barrera1, Deepak Haste2, Michael Renda2, Sudipto Ghoshal2, Jason H. Wong3
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), Orlando, Florida (4 December 2024)
The U.S. Marine Corps (USMC) has taken the initiative of introducing interactive learning experiences at its schoolhouses as a cost-effective and timesaving means to augment classroom instructions and physical equipment-training with immersive maintenance training and safety training in a simulated environment. However, the techniques for creating 3D models for immersive environments use Computer Aided Design (CAD) and graphics software, which demand significant manual effort, software skills, time, and financial investments. The USMC has recognized the need to rapidly build a repository of ready, reusable, and configurable 3D models of their assets in a scalable manner. Recent advances in generative artificial intelligence (AI) can fill this need by rapidly generating approximate but realistic 3D models from 2D pictures of equipment found in USMC training guides such as presentations and student handouts, thereby reducing program costs and accelerating student training.
In this paper, the research team presents a scalable and automated content-generation process that uses an ensemble of vision-based generative AI techniques, and a diverse web-sourced 3D-object dataset, to convert 2D images into 3D models with appropriate tradeoffs between desired quality and computational complexity. The research team will extend an existing foundational 2D-to-3D conversion-model trained with large and diverse web-scale data for “few-shot” transfer learning with domain-specific data. The 3D-content generation process will use open-source software and incorporate intuitive user-interfaces to minimize the need to learn machine learning (ML) or graphics programming. The resulting 3D objects can be imported into reusable libraries for use across various schoolhouse applications requiring immersive training content.
Furthermore, the team presents the results of performance experiments that convert images from a USMC schoolhouse course using techniques with varying degrees of complexity, and benchmark various vision-based AI/ML techniques for object fidelity and speed of conversion. The paper concludes with best practices and lessons learned from these content-conversion experiments.
1 Charles River Analytics
2 Qualtech Systems, Inc.
3 Naval Information Warfare Center Pacific
For More Information
To learn more or request a copy of a paper (if available), contact Eduardo Barrera.
(Please include your name, address, organization, and the paper reference. Requests without this information will not be honored.)
