Abstract
In this paper, we describe the partially observable Markov decision process (pomdp) approach to finding optimal or near-optimal control strategies for partially observable stochastic environments, given a complete model of the environment. The pomdp approach was originally developed in the operations research community and provides a formal basis for planning problems that have been of interest to the AI community. We found the existing algorithms for computing optimal control strategies to be highly computationally inefficient and have developed a new algorithm that is empirically more efficient. We sketch this algorithm and present preliminary results on several small problems that illustrate important properties of the pomdp approach. Introduction Agents that act in real environments, whether physical or virtual, rarely have complete information about the state of the environment in which they are working. It is necessary for them to choose their actions in partial ignorance and o...
Keywords
Partially observable Markov decision processObservableSketchComputer scienceMarkov decision processMathematical optimizationOptimal controlMarkov processProcess (computing)Control (management)Markov chainArtificial intelligenceAlgorithmMarkov modelMachine learningMathematics
Affiliated Institutions
Related Publications
Digital Natives, Digital Immigrants Part 1
The main purpose of this project is to develop mathematical competences through interactive educational environments of multi-device access in the ninth grade students of the In...
Wagner and Dollo: A Stochastic Duet by Composing Two Parsimonious Solos
New contributions toward generalizing evolutionary models expand greatly our ability to analyze complex evolutionary characters and advance phylogeny reconstruction. In this art...
Stochastic Petri net representation of discrete event simulations
In the context of discrete event simulation, the marking of a stochastic Petri net (SPN) corresponds to the state of the underlying stochastic process of the simulation and the ...
Comparison of Bayesian and maximum-likelihood inference of population genetic parameters
Abstract Comparison of the performance and accuracy of different inference methods, such as maximum likelihood (ML) and Bayesian inference, is difficult because the inference me...
MrBayes 3.2: Efficient Bayesian Phylogenetic Inference and Model Choice Across a Large Model Space
Since its introduction in 2001, MrBayes has grown in popularity as a software package for Bayesian phylogenetic inference using Markov chain Monte Carlo (MCMC) methods. With thi...
Publication Info
- Year
- 1994
- Type
- article
- Pages
- 1023-1028
- Citations
- 561
- Access
- Closed
External Links
Citation Metrics
561
OpenAlex
Cite This
Anthony R. Cassandra,
Leslie Pack Kaelbling,
Michael L. Littman
(1994).
Acting Optimally in Partially Observable Stochastic Domains.
, 1023-1028.