Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [bayesian-probability]

The tag has no usage guidance.

1
vote
0answers
25 views

Bayesian methods in online setting

Imagine the following (very concrete) model: We have a series of random variables $x_k$ with values in $\lbrace 0, 1\rbrace$. We assume $x_k \mid p_k \sim \operatorname{Alt}(p_k),$ where $p_0 \sim R(0,...
1
vote
0answers
21 views

Bayesian parameter estimation

I am generally not that knowledgeable for math, so if my question is too broad or inaccurate, please let me know. I am currently reading a paragraph of one paper (https://www.fil.ion.ucl.ac.uk/spm/...
0
votes
1answer
113 views

Shannon problem

Since a few days, I try in my research to model / formalize a source of Shannon a little weird, and I can't do it at all. First of all, I explain to you its operating principle and then I describe it ...
3
votes
1answer
218 views

Updating Geman and Geman (1984) on image restoration

I am reading the seminal paper Stuart Geman and Donald Geman, Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images, IEEE Transactions on Pattern Analysis and Machine ...
0
votes
0answers
52 views

Probability distribution of the Hadamard ratio of two degenerate multivariate Gaussian distributions?

This question pertains to the theory of Hadamard/elementwise functions of multivariate r.v.s/random vectors, which is unfortunately not a very popular topic: References for the theory of Hadamard ...
0
votes
0answers
26 views

Are Matérn class kernels universal kernels or not?

This is a question that I can't find the solution. I don't know it is a open question or it is a well-known result that can be attained from several lemmas. Here are the definition of Matérn class ...
0
votes
1answer
74 views

How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?

Problem Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$ Find the marginal distribution of each eigenvalue, using whatever you can. Background In my ...
0
votes
0answers
24 views

Dependencies for directed graphical models with both probabilistic and deterministic nodes

Based on new developments of variational Bayes recurrent neural networks, I have a question about dependencies over latent variables. I have no problems when there are no deterministic nodes. I can ...
6
votes
0answers
116 views

Existence of stick breaking representations for random measures

The Dirichlet process has a roughly size ordered representation in terms of beta random variables, called a stick-breaking representation (Sethuraman, 1994). Similar results hold for the beta process, ...
2
votes
2answers
142 views

Quantifying the effect of noise on the posterior variance in Gaussian processes / multivariate Gaussian vectors

Consider a real-valued Gaussian process $f$ on some compact domain $\mathcal{X}$ with mean zero and covariance function $k(x,x') \in [0,1]$ (also known as the kernel function). This question concerns ...
1
vote
0answers
35 views

RMHMC sampling in non-parametric setup

The aim is to sample distributions using Fisher information (as mass matrix in Hamiltonian MCMC sampling). Details can be found in http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.190.580&...
1
vote
1answer
54 views

A problem with elementary inequality involving probabilities and Brier scoring rule

I am trying to prove certain relations between certain values of the so called Brier inaccuracy measure (Brier scoring rule). Given a vector $p = (p_1, \ldots p_n)$, where $p_1 + \ldots p_n = 1$ and $...
0
votes
1answer
61 views

Learning a Gaussian from noisy observations

Is it possible to learn a distribution over the parameters ($K=\Sigma^{-1}$ and $\mu$) of a Gaussian from noisy measurements of $X$? (Starting with some appropriate prior over the parameters) I know ...
3
votes
1answer
111 views

Multivariate normal concentration

If $X\sim N(0,\Sigma)$ for some $d$-dimensional normal distribution, then $X = \Sigma^{1/2} Z$ where $Z\sim (0,I)$. How to compute the following quantity? $$ \operatorname{var} (X^T X) = \...
1
vote
1answer
79 views

Accounting for unobserved events in baysian learning

I wanted to use Bayes theorem to help me automate the task of deciding if I should ignore events, but I am not sure how to update the posterior if I do The simple story goes like this: An event $y_i$...
1
vote
1answer
122 views

convergence of Bayesian posterior with non iid data

Let $(\epsilon_t)_t$ be a sequence of iid random variables, distributed according to the density $f:\mathbb{R}\to (0,\infty)$ and $$ x_t = q( \theta^\star, x_1,x_2, \ldots, x_{t-1}) + \epsilon_t \,. ...
1
vote
0answers
34 views

Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
0
votes
0answers
82 views

Exploiting conditional independence for inference in Bayesian networks

How is conditional independence used for making probabilistic inference in Bayes networks easier or more efficient? For example, given the following Bayes network: Let's say I want to compute ...
1
vote
1answer
40 views

Bayesian estimation with lower dimensional prior

Let a statistical model of a random variable $X$ with parameter $\theta \in R^m$ be represented by a density function $p(X=x|\theta)$. Assume that the prior, $q(\cdot)$, is on a lower dimensional ...
2
votes
1answer
102 views

Adaptive priors

A lot of recent literature in Bayesian approach to inverse problems involves Adaptive priors, i.e - priors that depend on noise level. A lot of articles deal with optimization of contraction rates ...
2
votes
2answers
303 views

Bayes statistics precisely formulated

I am trying to learn something about Bayesian statistics, however, I am struggling already with the simplest equations and, moreover, with the very basic questions: What are we given? What is our goal?...
8
votes
1answer
287 views

Base schemes and Bayesian priors

One of Grothendieck's dicta about algebraic geometry is to consider "the relative situation", where one doesn't consider the category of schemes but of schemes over a fixed base scheme. In Bayesian ...
2
votes
0answers
49 views

A canonical example of the non-existence of predictive probability distribution

Section 3 of Fortini et al. (2000) states that Given $(X^\infty, \mathcal X^\infty,P)$, a predictive probability distribution of $x_n$ given $(x_1, \dots, x_{n-1})$ with respect to $P$ need not ...
-1
votes
1answer
275 views

Orthogonal decomposition of conditional expectations

Suppose I have a random variable $x$ and a set of conditional distributions on $x$. Here is an example where the conditionals are nested: $$q_1 := E(x|y_1), \quad q_2 := E(x|y_1,y_2),\quad q_3 := E(x|...
7
votes
1answer
594 views

Rate of convergence of Bayesian posterior

Suppose a data generating process (DGP) is parameterized by some unknown parameter $\theta_0$, say $P_{\theta_0}$, and we want to estimate the value of $\theta_0$ using Bayesian method. Let $\pi(\...
9
votes
1answer
259 views

In what sense is the Bayesian posterior mean a “convex combination”?

I asked this on math.stackexchange with no response, I'm hoping someone here might have something. Suppose I want to estimate $x \in \mathbb{R}^n$ from two signals with zero mean, normally ...
1
vote
2answers
609 views

Probability spaces involved in using Bayesian Inference

I am currently reading "Statistical and Inductive Inference by Minimum Message Length" by C.S. Wallace. In this, Wallace gives a fairly informal account of Bayesian Inference which, in the case ...
1
vote
0answers
87 views

Simultaneous multiple perturbations in Markov chain Monte Carlo

I'm coding a McMC algorithm for geophysical applications. Using the Metropolis-Hastings scheme to accept/reject the proposed models is smth that i thought i completely understood, but i don't. To be ...
2
votes
1answer
79 views

What is the problem with this model parameter estimation algorithm?

In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is $$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$ The standard way to estimate $\theta$ ...
3
votes
1answer
154 views

Parameter estimation using bayesian update on moduli space?

Scientists take a set of data points, say in ${\mathbb R}^2$, and, assuming that this data should fit a polynomial of degree $d$ (or an exponential, etc.), they estimate parameters. I would think ...
2
votes
0answers
175 views

Can truncated/non-smooth distributions be used as priors/posteriors in Variational Bayesian methods?

Variational Bayesian methods can sometimes be a good alternative to Markov Chain Monte Carlo numerical evaluation of probability distributions. They do this, as I understand it, by approximating the ...
-1
votes
2answers
120 views

What is the likehood function in the noise free observation case

In the nonlinear Bayesian Tracking problem, if we consider the noise exists only in the state equation : x[k] = f(x[k-1],v[k-1]) where vk-1 here is an iid process noise sequence And we suppose that ...
-1
votes
2answers
434 views

How to deal with this Chicken-And-Egg problem ?

Let's imagine designing an odds pattern for a game, in which players bet for win or lose. Suppose the probablity of winning is $p$, thus the probablity of losing is $1-p$. Now imagine $n_1$ people ...
1
vote
1answer
862 views

Exploiting conditional independence working with covariance matrices

I have a Bayesian network where the number of nodes is potentially large. I've conditioned on some of the nodes (observed data) and I'm trying to draw samples from the distribution remaining nodes (...
1
vote
1answer
120 views

Continuous-time Markov chain to sample Bayesian posterior distribution

Given a Bayesian network and evidence for the values of a subset of the variables, a standard question is to compute the posterior distribution on the remaining variables. The Gibbs sampling technique ...
2
votes
3answers
19k views

A “simple” explanation of the concept of D-separation in a Bayesian Network?

Hello everyone. I'm looking for a "simple" explanation of the concept of D-separation in a Bayesian Network. As far as I know the definition is "two variables (nodes) in the network are D-Separated ...
1
vote
1answer
323 views

Conditional probability and independence

Suppose that we have vectors of events $\{H_1,...,H_n\}$ and $\{D_1,...,D_m\}$. Consider the following two sets of conditions: Condition set 1 (1) $P(H_i H_j)=0$ for any $i\neq j$ and $\sum_iP(H_i)=...
2
votes
0answers
897 views

Estimating Wiener process parameters

Consider a Wiener process with zero drift, infintesimal variance $\sigma^2$, and an unknown starting value $\nu$. That is, \begin{align} Y_t \sim \mathcal{N}(\nu, t\sigma^2). \end{align} Now, ...
3
votes
1answer
3k views

Derivatives of conditional expectations

Let $X$, $Y$ and $Z$ be independent, real-valued random variables, probably with continuous density functions. Define $A = X + Y$ and $B = X + Z$. Consider the regular conditional expectation $E_Y(a,...
5
votes
1answer
1k views

What can be said about an infinite linear chain of conjugate prior distributions?

We can sample a discrete value from the multinomial distribution. We can also sample the parameters of the multinomial distribution from its conjugate prior the dirichlet distribution. Since the ...
-4
votes
2answers
931 views

In Bayesian statistics, must I use a marginalized prior in conjunction with a marginalized distribution?// [closed]

Suppose I have some sampling distribution g(x,y,z) which has been marginalized over some variables (say y and z) giving us the marginal distribution which we'll call gx(x). Suppose I now wish to use ...
4
votes
6answers
1k views

Are all probabilities conditional probabilities? [closed]

We know that $P(A\mid B) = \frac{P(A \cap B)}{P(B)}$. So $P(B) = P(A\mid B)P(A \cap B)$. Thus are all probabilities conditional probabilities? Can one make a probability more accurate by introducing a ...
5
votes
3answers
1k views

Probability estimates for pairwise majority votes

This is related to the rank aggregation question I asked previously. I have items $I_1, \ldots, I_N$ and the observations of a number of pairwise trials which pit pairs $I_i$ and $I_j$ against ...