This paper presents a novel approach to detect structural damage based on combining independent component analysis (ICA) extraction of time domain data and artificial neural networks (ANN). The advantage of using time history measurements is that the original vibration information is used directly. However, the volume of data, measurement noise, and the lack of reliable feature extraction tools are the major obstacles. To circumvent them, the independent component analysis technique is applied to represent the measured data with a linear combination of dominant statistical independent components and the mixing matrix [A]. Such a representation captures the essential structure of the measured vibration data. The vibration features represented by the mixing matrix provide the relationship between the measured vibration response and the independent components and are then employed to build the simplified neural network model for damage detection. Two examples are included to demonstrate the effectiveness of the method. First, a truss structure with simulated displacement data was used, and the results show that healthy and damage states located in the nine elements may be classified. Second, a bookshelf structure together with measured time history data from 24 piezoelectric single axis accelerometers was used to demonstrate the approach on a physical structure. The results show the successful detection of the undamaged and damaged states with very good accuracy and repeatability.