Workshop on Perception and Navigation for Autonomous Heavy Machines in Shared Environments

Abstract:"Planning and control of mobile machinery in unstructured and dynamic environments is a difficult problem regardless of the level of automation. Machines used for construction, mining and forestry have the task of physically manipulating the environment in various ways according to a well-defined goal, e.g., transforming the terrain to a new state; loading, transporting, and, off-loading objects of a certain kind; or subdividing or merging materials into new structures. This should be executed with a certain error tolerance, minimal use of time and energy, and without causing damage to the environment. To succeed in this, it is important to have good information about the surrounding environment, e.g., in the form of a semantic 3D map including information physical properties of objects and materials. In general, many vehicles operate simultaneously in these environments. This is a complicating factor but also offers opportunities. A machine can benefit from observations generated from another machine's interaction with the environment. A machine can improve, or worsen, the conditions for another machine to perform its tasks. We therefore develop solutions to estimate physical states and properties of materials and objects in the environment of mobile equipment. The world state is represented as a 3D scene with object and material properties that supports both physics-based forward simulation, motion planning using inverse dynamics or a learned world model, and generation of synthetic sensor data for model inference. We use OpenPLX for representing these world states in an interoperable format. Demonstrations include examples from forestry and earthmoving operations involving handling of logs, rocks, and deformable terrain. "

Abstract:"To achieve robust off-road navigation, there's an increasing demand for learning-based approaches capable of generalizing beyond trained situations. However, the dynamic and highly unstructured characteristics of off-road environments invariably result in distribution shifts, which can lead to unreliable outputs from learned models. This talk will introduce key methodologies designed to address the distribution shift challenge in robust off-road navigation. Specifically, we will explore how robust perception and planning can be effectively combined with learning-based adaptation in unstructured environments. "

Abstract:"No research algorithm or concept fully survives contact with the "real world", and this has been our experience in a number of projects translating world class fundamental research in perception and localization into deployable technology on autonomous vehicles, both on- and off-road. A flagship example of this has been our work in localization for underground mining vehicles, which, along with other projects in on-road autonomous vehicles, have provided many valuable insights that are worth sharing and discussing with the research community. These insights include: the limitations of "gold standard" performance metrics for localization and perception in predicting a technique's real-world utility; the underappreciated role of introspection - systems that "know when they don't know"; and the need for a much more sophisticated and nuanced understanding of errors and their highly variable significance across space, time and task. I'll illustrate these key concepts using concrete examples from our industry projects in these areas, and propose some paths forward for the research sector that will better link what is rewarded and celebrated in the research community with its likely eventual utility in deployed robots and systems."

Abstract:"This presentation will share insights from GIM Robotics' work in developing and deploying perception and navigation solutions for autonomous heavy machinery across multiple use cases. The discussion will begin by outlining the primary motivations and key challenges associated with creating autonomous systems for shared environments. Subsequently, we will showcase a ew success stories from diverse industries, demonstrating the real-world impact of our technology."

Abstract:"The role of robotics in all fields involving human presence is increasingly evident, both in the agriculture and construction sectors. Adopting and integrating advances in robotics into heavy machines for efficient large-scale agriculture and construction is challenging, particularly when striving to safeguard the environment and being conscious of sustainability. This talk presents insights and challenges in the research and development of YANMAR’s robotics technologies for perception-driven human assistance and autonomy, aimed at the noble pursuit of a sustainable future for humanity.  "

Abstract:"Autonomous off-road driving has broad applications in agriculture, mining, search and rescue, exploration, and defense. While off-road has many similarities to urban areas, a major difference is the lack of an obstacle/no obstacle dichotomy. That is, in off-road scenarios, not all objects are obstacles, and identifying which objects are traversable in a reliable way is critical. Our research covers a wide range of topics that aim at expanding the robot’s capability as well as improving its robustness in challenging environments. We utilize modern machine learning techniques while eliminating the exhausting hand-labeling process. Specifically, we explore self-supervised learning and learning-from-demonstration to understand the terrain traversability cost and vehicle dynamics from large-scale interaction data, and online adapt the cost and dynamics model to overcome the out-of-distribution failures. Our system doesn’t require human-labeled data. Instead, it relies on its own experiences of interacting with the environment, while being aware of the uncertainty in each model, and online adapts the model in novel situations. "