Accueil

One-Counter Markov Decision Processes.

Mon, 09/14/2009 - 09:02 — SomNambul
Authors: Tomáš Brázdil, Václav Brožek, Kousha Etessami, Antonín Kučera, Dominik Wojtczak
Subjects: Computer Science and Game Theory
link: http://arxiv.org/abs/0904.2511
Abstract

We study the computational complexity of central analysis problems for
One-Counter Markov Decision Processes (OC-MDPs), a class of finitely-presented,
countable-state MDPs. OC-MDPs are equivalent to a controlled extension of
(discrete-time) Quasi-Birth-Death processes (QBDs), a stochastic model studied
heavily in queueing theory and applied probability. They can thus be viewed as
a natural ``adversarial'' version of a classic stochastic model. Alternatively,
they can also be viewed as a natural probabilistic/controlled extension of
classic one-counter automata. OC-MDPs also subsume (as a very restricted
special case) a recently studied MDP model called ``solvency games'' that model
a risk-averse gambling scenario. Basic computational questions about these
models include ``termination'' questions and ``limit'' questions, such as the
following: does the controller have a ``strategy'' (or ``policy'') to ensure
that the counter (which may for example count the number of jobs in the queue)
will hit value 0 (the empty queue) almost surely (a.s.)? Or that it will have
infinite limsup value, a.s.? Or, that it will hit value 0 in selected terminal
states, a.s.? Or, in case these are not satisfied a.s., compute the maximum
(supremum) such probability over all strategies. We provide new upper and lower
bounds on the complexity of such problems. For some of them we present a
polynomial-time algorithm, whereas for others we show PSPACE- or BH-hardness
and give an EXPTIME upper bound. Our upper bounds combine techniques from the
theory of MDP reward models, the theory of random walks, and a variety of
automata-theoretic methods.

cheap louboutins

Thu, 11/24/2011 - 13:19 — dfa

One of the best seller shoes is louboutin shoes store. It is known to all that this brand christian louboutin on sale are so popular that almost everyone knows christian louboutin uk them. Especially the red sole high louboutin pumps online heels. Although this brand heels are designed discount christian louboutin shoes with high heel, but they very comfortable. So many ladies are fond of purchasing cheap louboutins sale.

http://www.louboutins-shop.com/

Thu, 11/24/2011 - 13:28 — dfa

The designer of discount christian louboutin shoes is the most understand woman's heart. He wanted to support christian louboutin online ladies transform or develop their identity through his cheap louboutin shoes designs. I have to say his design have aggressive discount louboutins succinct and suggestive emotions. This louboutin shoes sale Over The Knee Boots is a classique collection christian louboutin discount shoes.

http://www.cheapshoes-louboutins.com/

Thu, 11/24/2011 - 13:36 — dfa

Now matter where you are, if you wear a pair of christian louboutin boots Over The Knee Boots, you must be the most bright christian louboutin discount shoes beautiful woman, people will pay their cheap louboutin shoes attention on you. High quality with these nice christian louboutin on sale shoes, they won't hurt your feet when wearing. Get a pair of discount christian louboutin shoes from our sales online with low price, it will be your best christian louboutin sneakers decision.

http://www.louboutins4fashion.com/

Thu, 11/24/2011 - 13:43 — dfa

For all ladies, the newest christian louboutin shoes are their favorites and dreams. This design christian louboutin sale shoes are famous for the discount christian louboutin pumps style, excellence and comfort. Such sexy christian louboutin high heels shoes are welcomed by the world, and there have more and more people are crazy for discounted christian louboutin shoes them. It seems that choose cheap louboutin shoes are the best decision.

http://www.gianmarcolorenzishoes-online.com/

Thu, 11/24/2011 - 14:05 — dfa

Many different kinds of ysl shoes on sale are designed for people. No matter what you like, our discount ysl shoes shop do everything possible to meet you. Whether the gianmarco lorenzi shoes on sale style or color,you can not hesitate to buy yves saint laurent shoes here.Their unique design will make you feel ysl shoes sale more confident and charming. Welcome to our shop to enjoy our best discount gianmarco lorenzi shoes and best service.

http://www.cheaplouboutins-online.com/

Thu, 11/24/2011 - 15:26 — dfa

Winter is cold, you need a warm winter discount christian louboutin heels shoes.Of course, this pair of cheap louboutin pumps shoes must be fashionable. Then, I think above of christian louboutin on sale will make you satisfied. This time, it is hot and discount sale on our christian louboutin shoes online shop, you can enjoy high quality discount louboutins and cheap price at the same time if you buy christian louboutin boots them.

http://www.discountshoes-louboutins.com/

Thu, 11/24/2011 - 15:41 — dfa

I believe, these cheap louboutin shoes at our online shop must can meet your discount christian louboutin shoes demands. They are with exquisite design, and sale at reasonable discounted louboutins store online price, you can rest assured to buy christian louboutin heels what you like. I guarantee, choose our christian louboutin pumps sale online shop to buy christian louboutin boots will become the most wise decision what you have made.

http://www.monclers-cheapjackets.com/

Thu, 11/24/2011 - 16:38 — dfa

Should you added moncler boots inside wanted colors, you’ll come to be get noticed amongst the crew discounted moncler vest accessories. Putting moncler down jackets together suitably formulated moncler winter jackets, they’re most certainly apt to enhance the benefit of the discount moncler character plus the benefit for sticking to your physique temperatures daily while using regular moncler wiki really needs.

http://www.monclersale-discountjackets.com

Thu, 11/24/2011 - 16:38 — dfa

This kind of discounted moncler coats is unbelievable insulation, addition of moncler online are the light weight that make you moncler wiki feel more comfort and soft. Look here, I think you shouldn't miss this amazing cheap moncler jackets on our store online. Trust me, you will be have a wonderful cheap moncler down jackets and warm winter in this year. With the low price on sale at our discount moncler vest Online.

http://www.discount-sheepskin-boots.com/

Thu, 11/24/2011 - 18:10 — dfa

When you wear this design ugg boots uk walk in the street,there is no doubt that all people's attention will focus on cheap ugg boots sale you. If you do not believe what I say, why not buy our discount uggs on sale to have a try? I am sure that once you put our ugg classic boots on, the unique fresh feeling will let you ugg boots sale amazed. You will be happy to wear this brand ugg online outlet in this season.