title: Partially observable Markov decision processes and performance sensitivity analysis.
abstract: The sensitivity-based optimization of Markov systems has become an increasingly important area. From the perspective of performance sensitivity analysis, policy-iteration algorithms and gradient estimation methods can be directly obtained for Markov decision processes (MDPs). In this correspondence, the sensitivity-based optimization is extended to average reward partially observable MDPs (POMDPs). We derive the performance-difference and performance-derivative formulas of POMDPs. On the basis of the performance-derivative formula, we present a new method to estimate the performance gradients. From the performance-difference formula, we obtain a sufficient optimality condition without the discounted reward formulation. We also propose a policy-iteration algorithm to obtain a nearly optimal finite-state-controller policy.
authors.affiliation: Department of Automation, University of Science and Technology of China, Hefei 230026, China. whylyj@ustc.edu
authors.forename: Yanjie
authors.initials: Y
authors.lastname: Li
authors.forename: Baoqun
authors.initials: B
authors.lastname: Yin
authors.forename: Hongsheng
authors.initials: H
authors.lastname: Xi
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D003198:Computer Simulation; D003657:Decision Making; D003661:Decision Support Techniques; D008390:Markov Chains; D015233:Models, Statistical; D010363:Pattern Recognition, Automated / Q000379:methods*; D012680:Sensitivity and Specificity