Memristors have uses as artificial synapses and perform well in this role in simulations with artificial spiking neurons. Our experiments show that memristor networks natively spike and can exhibit emergent oscillations and bursting spikes. Networks of near-ideal memristors exhibit behaviour similar to a single memristor and combine in circuits like resistors do. Spiking is more likely when filamentary memristors are used or the circuits have a higher degree of compositional complexity (i.e. a larger number of anti-series or anti-parallel interactions). 3-memristor circuits with the same memristor polarity (low compositional complexity) are stabilised and do not show spiking behaviour. 3-memristor circuits with anti-series and/or anti-parallel compositions show richer and more complex dynamics than 2-memristor spiking circuits. We show that the complexity of these dynamics can be quanti fied by calculating (using partial auto-correlation functions) the minimum order auto-regression function that could fit it. We propose that these oscillations and spikes may have similar phenomena to brainwaves and neural spike trains and suggest that these behaviours can be used to perform neuromorphic computation.