Wearable systems constitute a promising solution to the emerging challenges of healthcare provision, feeding machine learning frameworks with necessary data. In practice, however, raw data collection is expensive in terms of energy, and therefore imposes a significant maintenance burden to the user, which in turn results in poor user experience, as well as significant data loss due to improper battery maintenance. In this paper, we propose a framework for on-board activity classification targeting severely energy-constrained wearable systems. The proposed framework leverages embedded classifiers to activate power-hungry sensing elements only when they are useful, and to distil the raw data into knowledge that is eventually transmitted over the air. We implement the proposed framework on a prototype wearable system and demonstrate that it can decrease the energy requirements by one order of magnitude, yielding high classification accuracy that is reduced by approximately 5%, as compared to a cloud-based reference system.