IoTpacket2vec: Contrastive Representation Learning for IoT Packet Embeddings

The deep learning paradigm has revolutionised feature learning for various problems, taking raw inputs and transforming into dense feature vectors, known as an embeddings, for downstream tasks. Contrastive learning is a representation learning technique that can transform similar inputs so they are close in the feature space. Embeddings learned using this contrastive approach have been very successful for many subsequent tasks, such as sentences and images. For resource constrained devices, typical of Internet of Things (IoT) devices, features are often manually extracted from network headers for various tasks, including anomaly detection, flow identification, and traffic classification. Feature learning for network traffic has been researched but using pre-processed representations, such as textual. Employing text representations on resource constrained device is generally not feasible. Currently, there is no feature learning approach for network packets appropriate for resource constrained devices. We propose IoTpacket2vec, a contrastive learning approach to train an encoder to directly take the raw input packet and embed the packet's headers into a useful embedding. We mask ephemeral fields (e.g. addresses) and evaluate a convolutional neural network, recurrent neural network, and transformer encoders appropriate for IoT devices. We use a 6LoWPAN and Zigbee dataset for training the encoders. To evaluate the embeddings, we use a packet type classification task and demonstrate the embeddings have learned a compact and useful representation of the packet. We then quantise the models, significantly reducing memory requirements, and evaluate how the encoders perform. We find that the convolutional neural network results in the best combination of accuracy, embedding size, and inference time.