There are well-known examples of dynamical systems for which the Birkhoff averages with respect to a given observable along some or all of the orbits do not converge. It has been suggested that such orbits could be classified using higher order averages. In the case of a bounded observable, we show that a classical result of G.H. Hardy implies that if the Birkhoff averages do not converge, then neither do the higher order averages. If the Birkhoff averages do not converge then we may denote by [αk,βk] the limit set of the k-th order averages. The sequence of intervals thus generated is nested: [αk+1,βk+1]⊂[αk,βk]. We can thus make a distinction among non-convergent Birkhoff averages; either: B1. [image omitted] is a point B∞, or, B2. [image omitted] is an interval [α∞,β∞]. We give characterizations of the types B1 and B2 in terms of how slowly they oscillate and we give examples that exhibit both behaviours B1 and B2 in the context of full shifts on finite symbols and 'Bowen's example'. For finite full shifts, we show that the set of orbits with type B2 behaviour has full topological entropy.