Continual learning-based trajectory prediction with memory augmented networks
Document Type
Article
Publication Date
12-22-2022
Abstract
Forecasting pedestrian trajectories is widely used in mobile agents such as self-driving vehicles and social robots. Deep neural network-based trajectory prediction models precisely predict pedestrian trajectories after training. However, the prediction models fail to avoid catastrophic forgetting when the data distribution shifts during continual learning, making it incredible to deploy the models on agents in real environments. A continual trajectory prediction method with memory augmented networks, CLTP-MAN, is proposed to handle the catastrophic forgetting issue by introducing a memory augmented network with sparse experience replay. CLTP-MAN comprises an external memory module, a memory extraction module, and a trajectory prediction module. The external memory module contains prior knowledge useful for trajectory prediction. The memory extraction module can read or write the key-value memories with a trainable controller. Last, the trajectory prediction module performs long-trajectory prediction by introducing a multi-hop attention mechanism to extract pivotal information from the external memory. Meanwhile, the catastrophic forgetting issue is handled through sparse experience replay. The two benchmarking datasets ETH/UCY and SDD are reintegrated according to the needs for continual learning to conduct quantitative and qualitative evaluations. The results verify that benefiting from external memory and the multi-hop attention mechanism, CLTPMAN has better generalization than several mainstream methods. Sparse experience replay effectively reduces catastrophic forgetting, leading to reliable deployments on mobile agents. (c) 2022 Elsevier B.V. All rights reserved.
Keywords
Trajectory prediction, Multi -hop attention, Memory augmented neural networks, Continual learning, Catastrophic forgetting
Divisions
ai
Funders
National Key Research and Development Program, China (Grant No: 2018AAA0100800),Key Research and Development Program of Jiangsu, China (Grant No: BK20192004),Postdoctoral Foundation of Jiangsu Province (Grant No: 2021K187B),National Postdoctoral General Fund (Grant No: 2021M701042),Jiangsu Provincial Science and Technology Department General project (Grant No: BK20221380),International and Exchanges of Changzhou (Grant No: CZ20200035 & NSFC62276090)
Publication Title
Knowledge-Based Systems
Volume
258
Publisher
Elsevier
Publisher Location
RADARWEG 29, 1043 NX AMSTERDAM, NETHERLANDS