A semi-supervised deep learning based video anomaly detection framework using RGB-D for surveillance of real-world critical environments

Video surveillance has become very important in the current era of smart cities. Large amounts of surveillance cameras are deployed at public and private places for surveillance of infrastructural property and public safety. These surveillance cameras generate huge amount of video data and it is impractical for a human observer to continuously monitor these long-hour videos manually and detect any unwanted or anomalous event. This paper presents a multi-modal semi-supervised deep learning based CNN-BiLSTM autoencoder framework to detect anomalous events in critical surveillance environments like Bank-ATMs. The significance of the framework is that it only requires weakly labelled normal video samples for training. We leverage the power of transfer learning by extracting important video features using a compact pretrained CNN to significantly reduce the computational complexity of training and detection. Moreover, due to the unavailability of any dataset for ATM surveillance in the public domain, we also contributed a unique RGB + D dataset for surveillance of ATMs. The proposed framework is tested on the collected RGB + D dataset and other real-world benchmark video anomaly datasets: Avenue and UCFCrime2Local. Results show that the proposed framework gives competitive results with other state-of-the-art methods and can be applied to both indoor and outdoor environments for detection of anomalies in real-world surveillance sites.