The quantification of sleep architecture has high clinical value for diagnostic purposes. While the clinical standard to assess sleep architecture is in-lab based polysomnography, higher ecological validity can be obtained with multiple sleep recordings at home. In this paper, we use a dataset composed of fifty sleep EEG recordings at home (10 per study participant for five participants) to analyze the sleep stage transition dynamics using Markov chain based modeling. The statistical analysis of the duration of continuous sleep stage bouts is also analyzed to identify the speed of transition between sleep stages. This analysis identified two types of NREM states characterized by fast and slow exit rates which from the EEG analysis appear to correspond to shallow and deep sleep respectively.