Elaboration on the policy improvement theorem for soft policies in reinforcement learning

Tim Kovacs

Elaboration on the policy improvement theorem for soft policies in reinforcement learning

Tim Kovacs

Intelligent Systems Laboratory

Research output: Working paper

Abstract

In section 5.4 of their book on reinforcement learning Sutton and Barto show that the policy improvement theorem applies to soft policies, that is, when making a soft policy greedier (but still soft) with respect to its Q-function we obtain an improved policy. I found this material difficult to follow and wrote this short document to elaborate on their proof. Familiarity with the material up until that section is assumed.

Translated title of the contribution	Elaboration on the policy improvement theorem for soft policies in reinforcement learning
Original language	English
Publisher	Department of Computer Science, University of Bristol
Publication status	Published - 2010

Bibliographical note

Other page information: -
Other identifier: 2001266

Access to Document

http://www.cs.bris.ac.uk/Publications/pub_master.jsp?id=2001266

Handle.net

Persistent link

Cite this

@techreport{1884ed4606ba4c839984456b7fed12be,

title = "Elaboration on the policy improvement theorem for soft policies in reinforcement learning",

abstract = "In section 5.4 of their book on reinforcement learning Sutton and Barto show that the policy improvement theorem applies to soft policies, that is, when making a soft policy greedier (but still soft) with respect to its Q-function we obtain an improved policy. I found this material difficult to follow and wrote this short document to elaborate on their proof. Familiarity with the material up until that section is assumed. ",

author = "Tim Kovacs",

note = "Other page information: - Other identifier: 2001266",

year = "2010",

language = "English",

publisher = "Department of Computer Science, University of Bristol",

type = "WorkingPaper",

institution = "Department of Computer Science, University of Bristol",

}

TY - UNPB

T1 - Elaboration on the policy improvement theorem for soft policies in reinforcement learning

AU - Kovacs, Tim

N1 - Other page information: - Other identifier: 2001266

PY - 2010

Y1 - 2010

N2 - In section 5.4 of their book on reinforcement learning Sutton and Barto show that the policy improvement theorem applies to soft policies, that is, when making a soft policy greedier (but still soft) with respect to its Q-function we obtain an improved policy. I found this material difficult to follow and wrote this short document to elaborate on their proof. Familiarity with the material up until that section is assumed.

AB - In section 5.4 of their book on reinforcement learning Sutton and Barto show that the policy improvement theorem applies to soft policies, that is, when making a soft policy greedier (but still soft) with respect to its Q-function we obtain an improved policy. I found this material difficult to follow and wrote this short document to elaborate on their proof. Familiarity with the material up until that section is assumed.

M3 - Working paper

BT - Elaboration on the policy improvement theorem for soft policies in reinforcement learning

PB - Department of Computer Science, University of Bristol

ER -

Elaboration on the policy improvement theorem for soft policies in reinforcement learning

Abstract

Bibliographical note

Access to Document

Handle.net

Fingerprint

Cite this