Timeline for Validating a probability density distribution forecast model for a Markov process
Current License: CC BY-SA 3.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 11, 2015 at 14:27 | comment | added | Guillaume Dehaene | Otherwise, I would do bayesian model comparaison against: a model with not time effect that fits every $X_t$ with a single distribution, a model with fits a Gaussian with the same mean and variance as your current prediction | |
| Aug 11, 2015 at 14:27 | comment | added | Guillaume Dehaene | Not that I know of, but that's not saying much ^^ Even if you can't rescale your predictions to a single shape, one thing you can do is check if some important statistics have been predicted well. Note $\mu_t, \sigma_t$ the prediction for the mean and std of $X_t$. Then the values $X_t - \mu_t$ should have empirical mean 0, and $(X_t - \mu_t)^2 / \sigma^2_t $ should have empirical mean 1. You can check those predictions (and similar predictions for the higher moments). If applicable, you can also check if the CLT applies and gives you the correct result. | |
| Aug 11, 2015 at 10:33 | comment | added | mt_christo | Guillaume, it's impossible to scale to similarity in my case, unfortunately - I want to track errors caused by wrong tail/mid scale shapes, and they are different even after being re-scaled (different tail power, for example). The family is still pretty simple (like Pareto - symmetric and good, with finite variances for my case, etc.), but rescaling is not applicable. Narrow case, but not as narrow as Gaussian. Do I understand it correctly, that there's no conservative probabilistic methodology around judgement of such varying distribution forecasts? | |
| Aug 10, 2015 at 19:59 | comment | added | Guillaume Dehaene | Can you maybe scale them back so that they are similar or something like that ? Otherwise, you ll have to go to model comparaison which will require a simple model to compare your proposal to. Look up bayesian model comparaison | |
| Aug 10, 2015 at 17:39 | comment | added | mt_christo | Distributions are a bit more complex, like power tail. Isn't there a probabilistic methodology in general ("good" enough though) case? | |
| Aug 10, 2015 at 15:37 | comment | added | Guillaume Dehaene | Well, if your predicted distros are all gaussian you could just rescale and recenter everybody and plot the histogram to check whether it s accurate. The next step is doing model comparison | |
| S Aug 10, 2015 at 15:13 | history | suggested | Jean Duchon | CC BY-SA 3.0 |
typo in title, and latexing formulas
|
| Aug 10, 2015 at 15:12 | review | Suggested edits | |||
| S Aug 10, 2015 at 15:13 | |||||
| Aug 10, 2015 at 8:49 | history | asked | mt_christo | CC BY-SA 3.0 |