Recognizing human activities with attention mechanism on multi-source sensor data

A smart home equipped with a diversity of multimodal sensors is a meaningful setting for acquiring the health status of its residents and improving their well-being. In recent years, sensor-based activity recognition has received growing research attention. However, the multi-modal nature of these sensor platforms raises great challenges with respect to the data fusion of the different sensor sources. To solve this problem, we present an activity recognition approach incorporating attention mechanism in this paper. A Convolutional Neural Network-based training framework is developed to extract representative features for activities. Specifically, we design two attention modules-channel-wise and temporal-wise modules to capture the interdependencies between channel and temporal dimensions of its convolutional features. We evaluate the attention-based approach on a real activity recognition challenge dataset. Experiments justify that the attention network-based feature fusion can effectively improve the activity recognition performance.