## Abstract

A class of Markov decision process is considered in which the boundedness of expected future costs is ensured by a natural form of termination, at least under some policies. Previous treatments of such problems have generally restricted attention to the case where the set of states is finite. In this paper, it is shown that all the results of the finite-state case hold when the set of states is a general Borel space, provided one makes the additional assumption that the optimal value function is bounded below. A sufficient condition is also given for the optimal value function to be bounded below which holds in particular if the set of states is countable.

Translated title of the contribution | An analysis of transient Markov decision processes |
---|---|

Original language | English |

Pages (from-to) | 603 - 621 |

Number of pages | 19 |

Journal | Journal of Applied Probability |

Volume | 43 (3) |

DOIs | |

Publication status | Published - Sep 2006 |