An analysis of transient Markov decision processes

HW James, EJ Collins

Research output: Contribution to journalArticle (Academic Journal)peer-review

7 Citations (Scopus)

Abstract

A class of Markov decision process is considered in which the boundedness of expected future costs is ensured by a natural form of termination, at least under some policies. Previous treatments of such problems have generally restricted attention to the case where the set of states is finite. In this paper, it is shown that all the results of the finite-state case hold when the set of states is a general Borel space, provided one makes the additional assumption that the optimal value function is bounded below. A sufficient condition is also given for the optimal value function to be bounded below which holds in particular if the set of states is countable.
Translated title of the contributionAn analysis of transient Markov decision processes
Original languageEnglish
Pages (from-to)603 - 621
Number of pages19
JournalJournal of Applied Probability
Volume43 (3)
DOIs
Publication statusPublished - Sep 2006

Bibliographical note

Publisher: Applied Probability Trust

Fingerprint Dive into the research topics of 'An analysis of transient Markov decision processes'. Together they form a unique fingerprint.

Cite this