Skip to main content
Capitalise title
Source Link
LSpice
  • 12.9k
  • 4
  • 45
  • 69

expected Expected gradient v.svs. gradient of expectation

Suppose a function $f(x): \mathbb R^d \mapsto \mathbb R^D$, and its stochastic approximator, $g(x; W): \mathbb R^d \mapsto \mathbb R^D$. Here $W$ is some random variable. TheThen $g(x; W)$ is unbiased in the sense that $$\mathbb E_W [g(x;W)] = f(x),$$ for any $x$.

I think the following two are not equal, but how to prove it? $$\mathbb E_W\left[\frac{dg(x;W)}{dx}\right] \text{ v.s. } \frac{df(x)}{dx}$$$$\mathbb E_W\left[\frac{dg(x;W)}{dx}\right] \text{ vs. } \frac{df(x)}{dx}.$$

expected gradient v.s. gradient of expectation

Suppose a function $f(x): \mathbb R^d \mapsto \mathbb R^D$, and its stochastic approximator, $g(x; W): \mathbb R^d \mapsto \mathbb R^D$. Here $W$ is some random variable. The $g(x; W)$ is unbiased in the sense that $$\mathbb E_W [g(x;W)] = f(x),$$ for any $x$.

I think the following two are not equal, but how to prove it? $$\mathbb E_W\left[\frac{dg(x;W)}{dx}\right] \text{ v.s. } \frac{df(x)}{dx}$$

Expected gradient vs. gradient of expectation

Suppose a function $f(x): \mathbb R^d \mapsto \mathbb R^D$, and its stochastic approximator, $g(x; W): \mathbb R^d \mapsto \mathbb R^D$. Here $W$ is some random variable. Then $g(x; W)$ is unbiased in the sense that $$\mathbb E_W [g(x;W)] = f(x),$$ for any $x$.

I think the following two are not equal, but how to prove it? $$\mathbb E_W\left[\frac{dg(x;W)}{dx}\right] \text{ vs. } \frac{df(x)}{dx}.$$

Source Link

expected gradient v.s. gradient of expectation

Suppose a function $f(x): \mathbb R^d \mapsto \mathbb R^D$, and its stochastic approximator, $g(x; W): \mathbb R^d \mapsto \mathbb R^D$. Here $W$ is some random variable. The $g(x; W)$ is unbiased in the sense that $$\mathbb E_W [g(x;W)] = f(x),$$ for any $x$.

I think the following two are not equal, but how to prove it? $$\mathbb E_W\left[\frac{dg(x;W)}{dx}\right] \text{ v.s. } \frac{df(x)}{dx}$$