Timeline for Estimating the entropy of the solution to an SDE
Current License: CC BY-SA 4.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 19, 2023 at 22:30 | comment | added | Thomas Kojar | The literature is huge because it really depends on the coefficients. I would start looking on other neural-networks articles that took the SDE route (eg. this is common in stochastic gradient descent). Also NN themselves have been used to study FP equations eg. proceedings.mlr.press/v145/zhai22a/zhai22a.pdf or cims.nyu.edu/~wtu1/papers/2020Chaos.pdf "Neural network representation of the probability density function of diffusion processes" contains many references | |
| Nov 19, 2023 at 22:25 | comment | added | user3002473 | @ThomasKojar any textbooks you'd recommend we look at to start digging? | |
| Nov 19, 2023 at 22:23 | comment | added | user3002473 | Either way thanks for the help! | |
| Nov 19, 2023 at 22:23 | comment | added | user3002473 | Interesting - actually I hadn't even considered that the SDE our neural network is approximating might not have a solution (well, I suppose it'd have a solution by virtue of the data "existing" - there's some sort of process that approximately fits it, or maximizes a likelihood, its just a matter of the capacity of the drift and diffusion networks). | |
| Nov 19, 2023 at 22:17 | comment | added | Thomas Kojar | If you have particular information on $\mu,\sigma$ eg. being some convolutions or some affine transformations, then we can try to get some bounds (put them in your post, not in the comments). But for the general setting, even uniqueness is not guaranteed eg. one often needs some Lipschitz/Hölder-regularity. | |
| Nov 19, 2023 at 22:14 | comment | added | Thomas Kojar | Not really. Most Fokker-Planks pdes don't even have any explicit solutions because they are highly nonlinear. However, Fokker-Plank pdes show up all over in physics, so there is a lot literature on doing simulations for it. Especially if you can get it in divergence-form. | |
| Nov 19, 2023 at 22:11 | comment | added | user3002473 | @ThomasKojar Unfortunately I think our particular "Fokker-Plank" is about as general as it gets - our drift and diffusion are just neural networks, hence why I was curious if there are things we can say in general. | |
| Nov 19, 2023 at 22:10 | history | edited | user3002473 | CC BY-SA 4.0 |
added 3 characters in body
|
| Nov 19, 2023 at 22:07 | comment | added | Thomas Kojar | Well,the entropy is just a particular integral over the transition density eg. $$E_{x_{0}}[g(X_{t})]=\int g(x) p_{t}(x_{0},x)dx$$, so yes a good approach is to approximate your particular Fokker-Plank. | |
| Nov 19, 2023 at 21:59 | history | asked | user3002473 | CC BY-SA 4.0 |