Perception and observation with networked multi-agent systems for automated shipping and harbor applications

• Umgebungswahrnehmung und -beobachtung mittels vernetzter Multi-Agent-Systeme für automatisierte Schifffahrts- und Hafenanwendungen

Lin, Jiaying; Abel, Dirk (Thesis advisor); Schön, Steffen (Thesis advisor)

Aachen : RWTH Aachen University (2022)
Dissertation / PhD Thesis

Dissertation, Rheinisch-Westfälische Technische Hochschule Aachen, 2022

Abstract

Environment perception is a fundamental element of automated vessels, especially in hightraffic areas such as harbors. The intelligent vessels should be aware of the situation, i.e., perceive and observe the objects in the environment, to avoid collisions while conducting highly automated tasks, such as autonomous docking and cooperative maneuvering. In the context of an intelligent harbor for future automated shipping, the vessels must be wirelessly connected for real-time information exchange and thus a robust cooperative localization and perception. Meanwhile, environment perception and observation based on sensor fusion have drawn much attention for maritime applications. However, few studies intelligently integrate various perception sensors, considering their characteristics. As for the perception and localization of networked multi-agent systems, few approaches exist to tackle the real world’s maritime scenarios. This thesis proposes a novel concept of environment perception and observation of multiagent systems for automated maritime applications. This approach uses Light Detection and Ranging (LiDAR) as a primary sensor, Automatic Identification Systems (AIS), and Radio Detection and Ranging System (radar) as assisting information sources. It consists of four functional modules: object detection, Multi-Object Tracking (MOT), static environment mapping, and networked localization. For detecting objects in the surroundings, a Convolutional Neural Network (CNN) is applied to recognize different patterns in LiDAR point clouds, extract objects from them, and generate bounding boxes for the detected objects. The detected objects are tracked by estimating their motion profile for possible collision avoidance in the MOT module, which integrates the detections from different perception sensor measurements. As for static mapping, several polygons are used to represent the static environment. In networked localization, the perception from a single vessel is integrated into a central server, such that more networked vessels can share and optimize their perception estimation. The proposed algorithms were evaluated with test drives conducted in Rostock harbor, Germany. The research vessels equipped with navigation and perception sensors carried out different driving scenarios, such as docking and maneuvering, in which promising performance of the algorithms was demonstrated. The proposed perception and observation of networked multi-agent systems present a possibility of highly accurate and robust surveillance in a connected, intelligent harbor.

Institutions

• Chair and Institute of Automatic Control [416610]