Communicating MDPs: Equivalence and LP properties

Jerzy A. Filar, Todd A. Schultz

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

It is shown that the communicating property of Markov Decision Processes (MDPs) is equivalent to satisfaction of sets of linear equations. A mapping between the 'multichain' and 'unichain' linear programs for undiscounted MDPs is developed by applying this equivalence. The mapping is used to prove that the simpler linear program solves communicating MDPs.

Original languageEnglish (US)
Pages (from-to)303-307
Number of pages5
JournalOperations Research Letters
Volume7
Issue number6
DOIs
StatePublished - Dec 1988

Fingerprint

Markov Decision Process
Equivalence
Linear Program
Linear equations
Linear equation
Markov decision process
Linear program

Keywords

  • irreducible Markov chains
  • linear programming

ASJC Scopus subject areas

  • Software
  • Management Science and Operations Research
  • Industrial and Manufacturing Engineering
  • Applied Mathematics

Cite this

Communicating MDPs : Equivalence and LP properties. / Filar, Jerzy A.; Schultz, Todd A.

In: Operations Research Letters, Vol. 7, No. 6, 12.1988, p. 303-307.

Research output: Contribution to journalArticle

@article{c8114151a08e4f95bc8f3878a4fe9620,
title = "Communicating MDPs: Equivalence and LP properties",
abstract = "It is shown that the communicating property of Markov Decision Processes (MDPs) is equivalent to satisfaction of sets of linear equations. A mapping between the 'multichain' and 'unichain' linear programs for undiscounted MDPs is developed by applying this equivalence. The mapping is used to prove that the simpler linear program solves communicating MDPs.",
keywords = "irreducible Markov chains, linear programming",
author = "Filar, {Jerzy A.} and Schultz, {Todd A.}",
year = "1988",
month = "12",
doi = "10.1016/0167-6377(88)90062-4",
language = "English (US)",
volume = "7",
pages = "303--307",
journal = "Operations Research Letters",
issn = "0167-6377",
publisher = "Elsevier",
number = "6",

}

TY - JOUR

T1 - Communicating MDPs

T2 - Equivalence and LP properties

AU - Filar, Jerzy A.

AU - Schultz, Todd A.

PY - 1988/12

Y1 - 1988/12

N2 - It is shown that the communicating property of Markov Decision Processes (MDPs) is equivalent to satisfaction of sets of linear equations. A mapping between the 'multichain' and 'unichain' linear programs for undiscounted MDPs is developed by applying this equivalence. The mapping is used to prove that the simpler linear program solves communicating MDPs.

AB - It is shown that the communicating property of Markov Decision Processes (MDPs) is equivalent to satisfaction of sets of linear equations. A mapping between the 'multichain' and 'unichain' linear programs for undiscounted MDPs is developed by applying this equivalence. The mapping is used to prove that the simpler linear program solves communicating MDPs.

KW - irreducible Markov chains

KW - linear programming

UR - http://www.scopus.com/inward/record.url?scp=0024128580&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0024128580&partnerID=8YFLogxK

U2 - 10.1016/0167-6377(88)90062-4

DO - 10.1016/0167-6377(88)90062-4

M3 - Article

AN - SCOPUS:0024128580

VL - 7

SP - 303

EP - 307

JO - Operations Research Letters

JF - Operations Research Letters

SN - 0167-6377

IS - 6

ER -