# Questions tagged [bayesian-probability]

The bayesian-probability tag has no usage guidance.

59
questions

**-1**

votes

**0**answers

82 views

### How to avoid using a probability distribution that doesn't exist?

I have this problem, of which I know the solution, but I'm looking for the mathematically proper way of writing it.
Say I have a (infinite) population of people, where each individual is labeled by ...

**4**

votes

**0**answers

95 views

### Convergence of the expectation of a random variable when conditioned on its sum with another, independent but not identically distributed

Suppose that for all $n \in \mathbf{N}$, $X_n$ and $Y_n$ are independent random variables with
$$X_n \sim \mathtt{Binomial}(n,1-q),$$
and
$$Y_n \sim \mathtt{Poisson}(n(q+\epsilon_n)),$$
where $q \in (...

**0**

votes

**1**answer

55 views

### Proving the existence of a symmetric Bayesian Nash equilibrium

I am currently faced with the following question:
Consider the public goods game. Suppose that there are $I > 2$ players and that
the public goods is supplied (with benefit of 1 for all players) ...

**0**

votes

**0**answers

30 views

### restriction of a formula with matrix inverse multiplied by a vector

I'm trying to reproduce a proof from this paper but I'm stuck in one point (Lemma 6). The general subject is bayesian model for multi-armed bandit problem solved with Thompson sampling.
I think I ...

**1**

vote

**0**answers

24 views

### Bayesian posterior consistency when prior distribution is induced by a diffusion

Let $\Pi_{b,\sigma}$ be a prior distribution on $\{z_t\}_{t<T}\in C_0[0,T]$ induced by the following diffusion:
\begin{align}
d\tilde z_t&=b(\tilde z_t,t)dt+\sigma(\tilde z_t,t) dW_t, ~...

**2**

votes

**0**answers

55 views

### Convergence of Bayesian posterior

Let $\Delta [0,1]$ denote the set of all probability distributions on the unit interval.
Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density ...

**1**

vote

**1**answer

53 views

### Conditional density for random effects prediction in GLMM

I am currently working on generalized linear mixed models (GLMM) and need some help concerning the prediction of the random effects. More specifically, I don't understand the given representation of ...

**0**

votes

**0**answers

43 views

### Minnesota prior with natural conjugate in Bayesian vector autoregression

"The natural conjugate prior does not allow us to use prior information of this form. It also does not allow us to use the Minnesota prior." (Koop & Korobilis, 2010, p. 9). They explain why this ...

**0**

votes

**1**answer

105 views

### Optimal solution to cross entropy loss in the continuous case

This could be a simple question but I don't have a satisfying answer.
Setup. Suppose that we have $K$ different classes, and consider cross entropy loss which maps a probability vector in the ...

**8**

votes

**3**answers

206 views

### What does the KL being symmetric tell us about the distributions?

Suppose two probability density functions, $p$ and $q$, such that $\text{KL}(q||p) = \text{KL}(p||q) \neq 0$. Intuitively, does that tell us anything interesting about the nature of these densities?

**0**

votes

**0**answers

45 views

### Can there be 'fixed parameters' in Bayesian statistics?

I was given the following assignment:
"The waiting time to see a teller at a bank between 12:00 PM and 1:00 PM on Fridays, denoted Y , is assumed to follow an exponential distribution, Exponential($\...

**2**

votes

**0**answers

40 views

### Have stick-breaking priors with non-iid atoms been considered, and if not, why not?

Roughly speaking, a stick-breaking prior is a random discrete probability measure $P$ on a measurable space $\mathcal X$ of the form
$$P=\sum_{j\ge1}w_j\delta_{\theta_j}$$
where $(w_j)_{j\ge1}$ is a ...

**0**

votes

**0**answers

17 views

### Quantiles of the Q values of an unknown MDP

Consider an MDP with $n$ states, $k$ actions, and discount factor $\gamma \in [0,1)$. We are uncertain of its reward function $R \in \mathbb{R}^{n \times k}$ and transition function $T \in \mathbb{R}^{...

**1**

vote

**1**answer

249 views

### Bayesian Inference with Student-t likelihood

Suppose I've observed $x$ from a Student-t distribution with unknown $\mu$, and I'd now like to infer $\mu$. Since the t-distribution isn't exponential family, there's no conjugate prior available, ...

**4**

votes

**1**answer

182 views

### Bounding the sensitivity of a posterior mean to changes in a single data point

There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...

**4**

votes

**0**answers

183 views

### Bayesian Networks and Polytree

I am a bit puzzled by the use of polytree to infer a posterior in a Bayesian Network (BN).
BN are defined as directed acyclic graphs. A polytree is DAG whose underlying undirected graph is a tree. ...

**2**

votes

**2**answers

146 views

### Parametrising a sparse orthogonal matrix

I need to find a way to parametrise a matrix that is both sparse (to some degree) and orthogonal, i.e., I am looking for a parametrisation that describes $A \in \mathbb{R}^{n\times m}$ such that $AA^𝑇...

**1**

vote

**0**answers

115 views

### Gaussian Integrals over Spheres

I'm after a reference for an integral. In particular, I am looking a way to approximate or calculate the following:
$$ \int \limits_{\| \theta \|_2 = 1} e^{(-(\theta - \mu)^T \Sigma (\theta - \mu))} ...

**1**

vote

**1**answer

87 views

### The expectation of binary logistics regression with respect to Gaussian distribution

I am trying to compute the expectation of $g(s,x)=s \ln \sigma(x)+(1-s)\ln(1-\sigma(x))$ with respect to the normal distribution $\mathcal{N}(x;m,v)$, where we have $\sigma(x)=\frac{1}{1+e^{-x}}$. If ...

**1**

vote

**1**answer

95 views

### Bayesian methods in online setting

Imagine the following (very concrete) model: We have a series of random variables $x_k$ with values in $\lbrace 0, 1\rbrace$. We assume $x_k \mid p_k \sim \operatorname{Alt}(p_k),$ where $p_0 \sim R(0,...

**1**

vote

**0**answers

30 views

### Bayesian parameter estimation

I am generally not that knowledgeable for math, so if my question is too broad or inaccurate, please let me know.
I am currently reading a paragraph of one paper (https://www.fil.ion.ucl.ac.uk/spm/...

**0**

votes

**1**answer

129 views

### Shannon problem

Since a few days, I try in my research to model / formalize a source of Shannon a little weird, and I can't do it at all. First of all, I explain to you its operating principle and then I describe it ...

**3**

votes

**1**answer

304 views

### Updating Geman and Geman (1984) on image restoration

I am reading the seminal paper
Stuart Geman and Donald Geman, Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images, IEEE Transactions on Pattern Analysis and Machine ...

**0**

votes

**1**answer

81 views

### How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?

Problem
Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$
Find the marginal distribution of each eigenvalue, using whatever you can.
Background
In my ...

**6**

votes

**0**answers

164 views

### Existence of stick breaking representations for random measures

The Dirichlet process has a roughly size ordered representation in terms of beta random variables, called a stick-breaking representation (Sethuraman, 1994). Similar results hold for the beta process, ...

**2**

votes

**2**answers

213 views

### Quantifying the effect of noise on the posterior variance in Gaussian processes / multivariate Gaussian vectors

Consider a real-valued Gaussian process $f$ on some compact domain $\mathcal{X}$ with mean zero and covariance function $k(x,x') \in [0,1]$ (also known as the kernel function). This question concerns ...

**1**

vote

**0**answers

40 views

### RMHMC sampling in non-parametric setup

The aim is to sample distributions using Fisher information (as mass matrix in Hamiltonian MCMC sampling). Details can be found in http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.190.580&...

**1**

vote

**1**answer

58 views

### A problem with elementary inequality involving probabilities and Brier scoring rule

I am trying to prove certain relations between certain values of the so called Brier inaccuracy measure (Brier scoring rule).
Given a vector $p = (p_1, \ldots p_n)$, where $p_1 + \ldots p_n = 1$ and $...

**0**

votes

**1**answer

77 views

### Learning a Gaussian from noisy observations

Is it possible to learn a distribution over the parameters ($K=\Sigma^{-1}$ and $\mu$) of a Gaussian from noisy measurements of $X$? (Starting with some appropriate prior over the parameters)
I know ...

**3**

votes

**2**answers

255 views

### Multivariate normal concentration

If $X\sim N(0,\Sigma)$ for some $d$-dimensional normal distribution, then $X = \Sigma^{1/2} Z$ where $Z\sim (0,I)$. How to compute the following quantity?
$$
\operatorname{var} (X^T X)
=
\...

**1**

vote

**1**answer

97 views

### Accounting for unobserved events in baysian learning

I wanted to use Bayes theorem to help me automate the task of deciding if I should ignore events, but I am not sure how to update the posterior if I do
The simple story goes like this:
An event $y_i$...

**1**

vote

**1**answer

169 views

### convergence of Bayesian posterior with non iid data

Let $(\epsilon_t)_t$ be a sequence of iid random variables, distributed according to the density $f:\mathbb{R}\to (0,\infty)$ and
$$
x_t = q( \theta^\star, x_1,x_2, \ldots, x_{t-1}) + \epsilon_t \,.
...

**1**

vote

**0**answers

34 views

### Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models.
Consider a hidden Markov model (HMM) with ...

**0**

votes

**0**answers

147 views

### Exploiting conditional independence for inference in Bayesian networks

How is conditional independence used for making probabilistic inference in Bayes networks easier or more efficient?
For example, given the following Bayes network:
Let's say I want to compute ...

**1**

vote

**1**answer

40 views

### Bayesian estimation with lower dimensional prior

Let a statistical model of a random variable $X$ with parameter $\theta \in R^m$ be represented by a density function $p(X=x|\theta)$. Assume that the prior, $q(\cdot)$, is on a lower dimensional ...

**2**

votes

**1**answer

193 views

### Adaptive priors

A lot of recent literature in Bayesian approach to inverse problems involves Adaptive priors, i.e - priors that depend on noise level. A lot of articles deal with optimization of contraction rates ...

**2**

votes

**2**answers

335 views

### Bayes statistics precisely formulated

I am trying to learn something about Bayesian statistics, however, I am struggling already with the simplest equations and, moreover, with the very basic questions: What are we given? What is our goal?...

**8**

votes

**1**answer

297 views

### Base schemes and Bayesian priors

One of Grothendieck's dicta about algebraic geometry is to consider "the relative situation", where one doesn't consider the category of schemes but of schemes over a fixed base scheme.
In Bayesian ...

**2**

votes

**0**answers

52 views

### A canonical example of the non-existence of predictive probability distribution

Section 3 of Fortini et al. (2000) states that
Given $(X^\infty, \mathcal X^\infty,P)$, a predictive probability distribution of $x_n$ given $(x_1, \dots, x_{n-1})$ with respect to $P$ need not ...

**-1**

votes

**1**answer

321 views

### Orthogonal decomposition of conditional expectations

Suppose I have a random variable $x$ and a set of conditional distributions on $x$. Here is an example where the conditionals are nested:
$$q_1 := E(x|y_1), \quad q_2 := E(x|y_1,y_2),\quad q_3 := E(x|...

**7**

votes

**1**answer

834 views

### Rate of convergence of Bayesian posterior

Suppose a data generating process (DGP) is parameterized by some unknown parameter $\theta_0$, say $P_{\theta_0}$, and we want to estimate the value of $\theta_0$ using Bayesian method. Let $\pi(\...

**9**

votes

**1**answer

302 views

### In what sense is the Bayesian posterior mean a “convex combination”?

I asked this on math.stackexchange with no response, I'm hoping someone here might have something.
Suppose I want to estimate $x \in \mathbb{R}^n$ from two signals with zero mean, normally ...

**1**

vote

**2**answers

731 views

### Probability spaces involved in using Bayesian Inference

I am currently reading "Statistical and Inductive Inference by Minimum Message Length" by C.S. Wallace. In this, Wallace gives a fairly informal account of Bayesian Inference which, in the case ...

**1**

vote

**0**answers

87 views

### Simultaneous multiple perturbations in Markov chain Monte Carlo

I'm coding a McMC algorithm for geophysical applications.
Using the Metropolis-Hastings scheme to accept/reject the proposed models is smth that i thought i completely understood, but i don't. To be ...

**2**

votes

**1**answer

81 views

### What is the problem with this model parameter estimation algorithm?

In a statistical model with parameters $\theta$ and unobserved laten variables $Z$, the model likelihood is
$$L(\theta;X)=Pr(X|\theta)=\sum_ZPr(X,Z|\theta)$$
The standard way to estimate $\theta$ ...

**3**

votes

**1**answer

161 views

### Parameter estimation using bayesian update on moduli space?

Scientists take a set of data points, say in ${\mathbb R}^2$, and, assuming that this data should fit a polynomial of degree $d$ (or an exponential, etc.), they estimate parameters.
I would think ...

**2**

votes

**0**answers

210 views

### Can truncated/non-smooth distributions be used as priors/posteriors in Variational Bayesian methods?

Variational Bayesian methods can sometimes be a good alternative to Markov Chain Monte Carlo numerical evaluation of probability distributions. They do this, as I understand it, by approximating the ...

**-1**

votes

**2**answers

147 views

### What is the likehood function in the noise free observation case

In the nonlinear Bayesian Tracking problem, if we consider the noise exists only in the state equation : x[k] = f(x[k-1],v[k-1]) where vk-1 here is an iid process noise sequence
And we suppose that ...

**-1**

votes

**2**answers

453 views

### How to deal with this Chicken-And-Egg problem ?

Let's imagine designing an odds pattern for a game, in which players bet for win or lose.
Suppose the probablity of winning is $p$, thus the probablity of losing is $1-p$.
Now imagine $n_1$ people ...

**1**

vote

**1**answer

971 views

### Exploiting conditional independence working with covariance matrices

I have a Bayesian network where the number of nodes is potentially large. I've conditioned on some of the nodes (observed data) and I'm trying to draw samples from the distribution remaining nodes (...