# Tagged Questions

0answers
16 views

### Zeros of non-lipschitz functions (when noisy estimates are available only)

Given noisy (martingale difference) of a Lipschitz continuous function $f$ it is known how to compute zeros of it. It is the stochastic approximation approach (by Borkar, Kushner and Yin etc.). Is ...
0answers
18 views

### Examples of POMDPs where the actions impact the transitions of the underlying markov Chain

I am not sure if the following is a legitimate question for this board. I am looking for examples of Partially observed Markov decision processes (preferably infinite horizon, Discrete time, Discrete ...
1answer
124 views

### Supremum in a Markov chain model

A Markov chain $X$ with finite state space $\{1,2,\cdots,N\}$ is defined on a probability space $(\Omega, P, \mathcal{F})$ equiped with filtration $\{\mathcal{F}_t\}$. And we assume that we can reach ...
1answer
95 views

### Kalman filter with long term bias

I was reading about the Kalman filter and I do not understand how it should be used when our measurements have a long term offset like GPS location updates do. As I understand, the Kalman filter ...
1answer
248 views

### Anyone has Kushner's book “Introduction to stochastic control” 1971? I need a theorem from it

In a paper I'm reading, it refers to Theorem 8, Page 217 of the book "Introduction to Stochastic Control" H. J. Kushner, New York: Holt, Reinhart, and Winston 1971. Unfortunately I don't have it and ...
1answer
83 views

0answers
147 views

### stochastic control / geometric mean

Consider the following problem: Given $\Omega$ and $U$ two symmetric definite positive matrices, choose a matrix $K$ to minimize the expectation $x' \Omega x + x'K'UKx$ when $x$ follows the invariant ...
3answers
431 views

### convergence in distribution of stochastic gradient descent.

The stochastic gradient descent algorithm where only a noisy gradient (zero mean noise) is used to update current estimate is known to converge almost surely to the minimizer. However, if one is ...
2answers
1k views

### Commuting supremum and expectation

Given a one-parametric random function on a probability space $(\Omega,\mathcal F,\mathbb P)$: \$X:U\times\Omega\to \mathbb R \text{ and } (a,w)\mapsto X(a,w), \text{ with } \sigma(X(a))\subseteq ...