Online suspicious event detection in a constrained environment with RGB+D camera using multi-stream CNNs and SVM

Automated detection of human actions is very important for surveillance of government offices, Bank-ATMs, etc. to prevent crime and loss of property and lives of people. Currently, such environments are monitored by CCTV cameras which give only RGB frame of the scene. To detect unusual human actions, we propose to use a depth sensor, as it additionally provides depth and skeletal data of human being. This paper presents a 2-stage framework to detect a suspicious event and unsafe activity in an ATM as a use case of constrained environment. In the first stage, any suspicious event which arises due to unusual human actions is detected proactively in real-time. This is achieved by classifying motion representation images of RGB and Depth video segments using multi-stream CNNs (Convolutional Neural Networks). In the second stage, entire activity is analyzed as safe or unsafe by classifying spatiotemporal skeleton features through SVM (Support Vector Machine). Due to the unavailability of datasets for analyzing human actions in Bank-ATMs, we also contributed a unique RGB+D dataset by replicating activities in ATM-like environment. Experimental results show that our approach can detect suspicious event with 0.807 F_measure and classifies activities with 89.1% accuracy.