DSB seminar: Depth Video-based Robust Human Activity Recognition System

Abstract Human activity recognition from video has been getting considerable attentions by computer vision and artificial intelligence researchers due to its prominent applications in various fields such as video surveillance, human computer interactions, and smart home healthcare. For instance, activity recognition can be used in a surveillance environment to alert the related authority of potential dangerous behaviors. Similarly, the activity recognition can improve the human computer interaction (HCI) in an entertainment environment such as the automatic recognition of different player’s actions in a game to create an avatar to play on behalf for the player. Furthermore, the activity recognition can help the rehabilitation of patients in a healthcare system where patient’s action recognition can help to facilitate the rehabilitation processes. Basically, a video-based activity recognition system may have many prominent goals, one of which is to provide information based on people’s behavior to allow the system to proactively assist them with their tasks. A robust approach is discussed here for depth video-based human activity recognition using body joint-based features of depth body shapes. From depth video, different body parts of human activities are first segmented using a trained random forest. Then, spatial features consisting of the 3-D body joint pair angles, the mean of the depth values, the variance of the depth values, and the area of each segmented body part are combined with the motion features representing the magnitude and direction of each joint in the next frame to build spatiotemporal features in an activity image. The activity features are then further enhanced using generalized discriminant analysis to convert them to more robust ones. Finally, the features can be utilized for training activity models such as hidden Markov models or deep belief network that can be later used for recognition. The discussed approach shows superior recognition performance compared to other conventional activity recognition approaches.