Optical flow for human activity recognition: A systematic review from classical methods to deep learning

Abstract

Human Activity Recognition (HAR) from video has become increasingly important in applications ranging from surveillance and healthcare monitoring to humanrobot interaction. Optical flow, which describes apparent motion between consecutive frames, remains one of the most widely used temporal representations in video-based HAR. Despite this widespread use, the literature still offers limited systematic guidance on how different optical flow methods should be selected and integrated into activity-recognition pipelines. This review addresses that gap through a systematic analysis of 205 studies published between 2000 and 2025 and retrieved from IEEE, Elsevier, and Springer databases. The analysis focuses on four principal knowledge-based optical flow formulations: LucasKanade, HornSchunck, Farnebäck, and TV-L1. For each method, the review examines its combination with feature extraction strategies, classifier architectures, and benchmark datasets. The results reveal substantial methodological changes over the review period. LucasKanade dominated early work and appeared in 76 studies, whereas Farnebäck became more prominent in later periods, appearing in 53 studies. A clear shift from traditional machine learning to deep learning classifiers occurred after 2017: neural-network adoption increased from 13% in 20142017 to 51% in 20182020 and 63% in 20212025. Before 2018, histogram-based descriptors were the standard representation, but they were gradually replaced by the direct use of optical flow fields as inputs to convolutional and recurrent neural networks. TV-L1 showed particularly strong compatibility with deep learning architectures, with 15 of the 18 TV-L1 studies using neural-network classifiers. The review identifies persistent challenges, including computational efficiency for embedded deployment, robustness to environmental variation, and the lack of optical flow formulations designed specifically for activity recognition. A comparative analysis of 11 benchmark datasets is also provided to support evaluation under both controlled and real-world conditions. (c) 2026 The Authors