A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes

TitleA Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes
Publication TypeConference Paper
Year of Publication2006
AuthorsWolovick, N, Johr, S
Conference NameFormal Modeling and Analysis of Timed Systems, 4th International Conference, FORMATS 2006
PublisherSpringer
ISBN Number3-540-45026-2
AbstractContinuous-time Markov decision process are an important variant of labelled transition systems having nondeterminism through labels and stochasticity through exponential fire-time distributions. Nondeterministic choices are resolved using the notion of a scheduler. In this paper we characterize the class of measurable schedulers, which is the most general one, and show how a measurable scheduler induces a unique probability measure on the sigma-algebra of infinite paths. We then give evidence that for particular reachability properties it is sufficient to consider a subset of measurable schedulers. Having analyzed schedulers and their induced probability measures we finally show that each probability measure on the sigma-algebra of infinite paths is indeed induced by a measurable scheduler which proves that this class is complete.
DOI10.1007/11867340_25