Abstract

The benefits of automation are well known: dramatically lower direct labor costs with increased productivity, and improved fulfillment time and accuracy. Automation also lowers ancillary labor-related costs for areas such as safety, healthcare, employment litigation, and labor negotiation. The important benefits of automation come at a price, however, as the development time and cost to design these systems have also dramatically increased. Physical prototypes of complex systems like large-throughput warehouses, with the associated integration of equipment such as in-line sensors, computer vision, robotics, control software and other systems, as well as the evaluation of those systems with live human test subjects, is extremely costly, time consuming, and potentially unsafe. Presented is a method where complex systems and processes are developed using a virtual environment, commonly known as a Digital Twin. The twin replicates the proposed physical space, and includes all mechanical, robotic, sensor, vision, AI, and computer systems, while simultaneously incorporating real human action and interaction. Humans experience the system in virtual reality (VR), with concurrent motion capture driving their avatars in real time. The avatars interact with virtual objects and their physical “stand ins” which are also motion-tracked. All aspects of the system, including hardware, software, AI training data/models, and human interaction, are developed, tested, and optimized virtually using a software platform running on industrial GPUs such as the NVIDIA RTX 2080 Ti. The result is faster design discovery and iteration at a much lower cost. Once tested and optimized, the virtual system is deployed physically. Team collaboration is also improved, as key stakeholders in training, operations, safety, and other groups can join the environment virtually as needed, even if located thousands of miles apart. The presentation will provide examples of the development techniques and technologies that are used to create and optimize True Digital Twins. This includes multiple video examples of real life applications and their identical virtual twins in the robotic automation, retail, and medical fields.