Search Results

If you want to have $X_t$ as a "deformed" $W_t$ - at first I advise to assume $\sigma\neq 0$ a.s. Otherwise you will have some problems (really in such points you may have almost deterministic dynam …

Consider a real-valued Markov process $X$ with a transition density $f(x,y)$, i.e. $$ \mathsf P[X\in A|X_0 = x] = \int\limits_A f(x,y)\,dy. $$ For this process I want to find $$ u(x) = \mathsf P[X_n …

If one talks about homogeneous Markov diffusion $$ \mathrm dX_t = \mu(X_t)\mathrm dt+\sigma(X_t)\mathrm dw_t $$ with $\mu,\sigma$ sufficiently differentiable and of appropriate dimensions, there is ni …

Let $X$ be a strong Markov process in the continuous time with a state space $\mathbb R^n$. Consider a reachability problem for this process, i.e. $$ v(x):=\mathsf P_x(X_t\in A\text{ for some }0\leq t …

For a Markov process $X$ on the Polish space $\mathscr X$ its transition probability is given by $$ P(x,A) :=\mathsf P_x (X_1\in A) $$ and $X$ is time-reversible if there is a probability measure $\pi …

Let $P(x,A)$ be a stochastic kernel on a measurable space $(E,\mathcal E)$ and $G = \sum\limits_0^\infty P^n$ be its potential kernel. A $\sigma$-finite measure $\phi$ is called the irreducibility mea …

Consider a Markov Decision Process with Borel state space $X$ and Borel action space $U$, like the one defined in the book "Stochastic Optimal Control: Discrete-time case" by Bertsekas and Shreve. All …

Just for the sake of the question having an answer, the following paper by A. Maitra and W. Sudderth gives a very nice characterization of the value function for the i.o. event in the framework of gam …

Let $X$ and $Y$ be standard Borel spaces and let $J$ be an analytic subset of $X\times \mathcal P(Y)$ where $\mathcal P(\Omega)$ is a set of probability measures on a Borel space $\Omega$ endowed wit …

Let $X = (X_1,X_2)$ and $\hat X = (\hat X_1,\hat X_2)$ be two random variables where $X_i,\hat X_i$ are taking values over the Polish space $E_i$ endowed with their Borel $\sigma$-algebras, where $i=1 …

Let us consider a Markov Decision Process (MDP) with a Borel state space $X$. Often, the optimization problems over MDP involve optimization of some objectives dependent on the reward function $$ r: …

Before passing to question, let me briefly recap what's importance sampling of random variables is about. Suppose $\xi$ is a real-valued random variable with density $f$, and let $g:\Bbb R\to \Bbb R$ …

Let $X$ be a homogeneous Markov process in a continuous time with value in the set $E$. Suppose that for some $T>0,x\in E, A\subset E$ we have $$ P_x[X_t\in A] = 0 $$ for all $t\in [0,T]$ but $$ P_x[X …

Let $X$ be a Markov process (in continuous or discrete time) and define an event $$ R(T,A) = (\exists t\leq T: X_t \in A). $$ I have seen in one paper that $$ \Pr[R(\infty,A)] = \sup\limits_{\tau} \m …

We begin with example. For the Poisson process with an intensity $\lambda_1$ there is an equivalent change of measure which makes it intensity to $\lambda_2$. I would like to find the conditions whe …