Can today's global climate model ensembles characterize the 21st century climate in their own 'model-worlds'? This question is at the heart of how we design and interpret climate model experiments for both science and policy support. Using a low-dimensional nonlinear system that exhibits behaviour similar to that of the atmosphere and ocean, we explore the implications of ensemble size and two methods of constructing climatic distributions, for the quantification of a model's climate. Small ensembles are shown to be misleading in non-stationary conditions analogous to externally forced climate change, and sometimes also in stationary conditions which reflect the case of an unforced climate. These results show that ensembles of several hundred members may be required to characterize a model's climate and inform robust statements about the relative roles of different sources of climate prediction uncertainty.