# Questions tagged [bayesian-probability]

The bayesian-probability tag has no usage guidance.

68
questions

**3**

votes

**1**answer

59 views

### Gaussian process kernel parameter tuning

I am reading on gaussian processes and there are multiple resources that say how the parameters of the prior (kernel, mean) can be fitted based on data,specifically by choosing those that maximize the ...

**0**

votes

**0**answers

19 views

### Estimation of probability matrix from samples at different time intervals

I am given discrete-time Markov chain that evolves on a finite subset $\{1,\dots,n\}$. This Markov chain is time-homogeneous and has a transition matrix $P$ that I want to estimate.
Let $X_t$ be the ...

**0**

votes

**0**answers

25 views

### Integrating multinomial likelihood over intersection of two simplexes

I am trying to calculate this integral, coming from a multinomial likelihood with an extra condition ($\sum_{i=1}^n p_i w_i = 1$). Essentially it can be seen as integrating a Dirichlet pdf over a ...

**0**

votes

**1**answer

93 views

### CLT for random variables with positive support (e.g. exponential)

I have a bunch of iid $\{X_i\}$ with $X_i \sim \exp(\lambda)$ - let's say $\lambda = 1$. Now, classic version of CLT tells me:
\begin{equation}
\sqrt{n}\left(1-\bar{X}_n\right) \rightarrow \mathcal{N}\...

**0**

votes

**0**answers

34 views

### 2d interpolation minimizing the integral of the norm of the Hessian

It is well known that cubic interpolation is the solution of the interpolation problem that minimizes the integral of the square of the second derivative:
$$ min_{f \text{ s.t. } f(x_i)=y_i} \int (f''(...

**0**

votes

**1**answer

55 views

### Lower bound for reduced variance after conditioning

Let $X$ be a random variable with variance $\tau^2$ and $Y$ be another random variable such that $Y-X$ is independent of $X$ and has mean zero and variance $\sigma^2$. (One can think of $Y$ as a noisy ...

**-1**

votes

**1**answer

27 views

### Linear operator over a simplex space in a multinomial distribution parameter estimation problem

This is actually a variant of a well-known problem of how the parameters of a multinomial distribution can be estimated by maximum likelihood, and this arises from a final year project I undertook ...

**1**

vote

**1**answer

155 views

### Posterior expected value for squared Fourier coefficients of random Boolean function

Let $f : \{0, 1\}^{n} \rightarrow \{-1, 1\}$ be a Boolean function. Let the Fourier coefficients of this function be given by
$$ \hat f(z) = \frac{1}{2^{n}} \sum_{x \in \{0, 1\}^{n}} f(x)(-1)^{x \cdot ...

**1**

vote

**0**answers

53 views

### Bayesian inference of stochastically evolving model parameters

I have a question related to self-calibration in radio interferometry, but I will try to phrase it as generic as possible. I have a set of data points, $D = \{ d_{0, t_0}, d_{1, t_0}, ..., d_{M, t_0}, ...

**1**

vote

**1**answer

195 views

### Convolution of two Gaussian mixture model

Suppose I have two independent random variables $X$, $Y$, each modeled by the Gaussian mixture model (GMM). That is,
$$
f(x)=\sum _{k=1}^K \pi _k \mathcal{N}\left(x|\mu _k,\sigma _k\right)
$$
$$
g(y)=\...

**2**

votes

**0**answers

108 views

### Minimizing an f-divergence and Jeffrey's Rule

My question is about f-divergences and Richard Jeffrey's (1965) rule for updating probabilities in the light of partial information.
The set-up:
Let $p: \mathcal{F} \rightarrow [0,1]$ be a ...

**1**

vote

**0**answers

36 views

### Quantitative bounds on convergence of Bayesian posterior

Let $Y$ be a random variable in $[0,1]$, and let $X_1, X_2, \ldots$ be a sequence of random variables in $[0,1]$. Suppose that the $X_i$'s are conditionally i.i.d given $Y$ ; in other words, I'd like ...

**2**

votes

**0**answers

82 views

### Is there any good reference on the Bayesian view that can be helpful for reading papers on the number theory using heuristic arguments?

Nowadays there are many papers on the number theory using heuristics.
I have read some of them.
But I have no clear understanding of the Bayesian Probability(subjective probability).
The concept of ...

**4**

votes

**0**answers

107 views

### Convergence of the expectation of a random variable when conditioned on its sum with another, independent but not identically distributed

Suppose that for all $n \in \mathbf{N}$, $X_n$ and $Y_n$ are independent random variables with
$$X_n \sim \mathtt{Binomial}(n,1-q),$$
and
$$Y_n \sim \mathtt{Poisson}(n(q+\epsilon_n)),$$
where $q \in (...

**0**

votes

**1**answer

89 views

### Proving the existence of a symmetric Bayesian Nash equilibrium

I am currently faced with the following question:
Consider the public goods game. Suppose that there are $I > 2$ players and that
the public goods is supplied (with benefit of 1 for all players) ...

**0**

votes

**0**answers

33 views

### restriction of a formula with matrix inverse multiplied by a vector

I'm trying to reproduce a proof from this paper but I'm stuck in one point (Lemma 6). The general subject is bayesian model for multi-armed bandit problem solved with Thompson sampling.
I think I ...

**1**

vote

**0**answers

32 views

### Bayesian posterior consistency when prior distribution is induced by a diffusion

Let $\Pi_{b,\sigma}$ be a prior distribution on $\{z_t\}_{t<T}\in C_0[0,T]$ induced by the following diffusion:
\begin{align}
d\tilde z_t&=b(\tilde z_t,t)dt+\sigma(\tilde z_t,t) dW_t, ~...

**1**

vote

**0**answers

68 views

### Convergence of Bayesian posterior

Let $\Delta [0,1]$ denote the set of all probability distributions on the unit interval.
Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density ...

**1**

vote

**1**answer

61 views

### Conditional density for random effects prediction in GLMM

I am currently working on generalized linear mixed models (GLMM) and need some help concerning the prediction of the random effects. More specifically, I don't understand the given representation of ...

**0**

votes

**1**answer

206 views

### Optimal solution to cross entropy loss in the continuous case

This could be a simple question but I don't have a satisfying answer.
Setup. Suppose that we have $K$ different classes, and consider cross entropy loss which maps a probability vector in the ...

**8**

votes

**3**answers

242 views

### What does the KL being symmetric tell us about the distributions?

Suppose two probability density functions, $p$ and $q$, such that $\text{KL}(q||p) = \text{KL}(p||q) \neq 0$. Intuitively, does that tell us anything interesting about the nature of these densities?

**3**

votes

**0**answers

43 views

### Have stick-breaking priors with non-iid atoms been considered, and if not, why not?

Roughly speaking, a stick-breaking prior is a random discrete probability measure $P$ on a measurable space $\mathcal X$ of the form
$$P=\sum_{j\ge1}w_j\delta_{\theta_j}$$
where $(w_j)_{j\ge1}$ is a ...

**3**

votes

**1**answer

576 views

### Bayesian Inference with Student-t likelihood

Suppose I've observed $x$ from a Student-t distribution with unknown $\mu$, and I'd now like to infer $\mu$. Since the t-distribution isn't exponential family, there's no conjugate prior available, ...

**5**

votes

**1**answer

209 views

### Bounding the sensitivity of a posterior mean to changes in a single data point

There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...

**4**

votes

**0**answers

295 views

### Bayesian Networks and Polytree

I am a bit puzzled by the use of polytree to infer a posterior in a Bayesian Network (BN).
BN are defined as directed acyclic graphs. A polytree is DAG whose underlying undirected graph is a tree. ...

**2**

votes

**2**answers

254 views

### Parametrising a sparse orthogonal matrix

I need to find a way to parametrise a matrix that is both sparse (to some degree) and orthogonal, i.e., I am looking for a parametrisation that describes $A \in \mathbb{R}^{n\times m}$ such that $AA^𝑇...

**1**

vote

**0**answers

177 views

### Gaussian Integrals over Spheres

I'm after a reference for an integral. In particular, I am looking a way to approximate or calculate the following:
$$ \int \limits_{\| \theta \|_2 = 1} e^{(-(\theta - \mu)^T \Sigma (\theta - \mu))} ...

**1**

vote

**1**answer

92 views

### The expectation of binary logistics regression with respect to Gaussian distribution

I am trying to compute the expectation of $g(s,x)=s \ln \sigma(x)+(1-s)\ln(1-\sigma(x))$ with respect to the normal distribution $\mathcal{N}(x;m,v)$, where we have $\sigma(x)=\frac{1}{1+e^{-x}}$. If ...

**1**

vote

**1**answer

143 views

### Bayesian methods in online setting

Imagine the following (very concrete) model: We have a series of random variables $x_k$ with values in $\lbrace 0, 1\rbrace$. We assume $x_k \mid p_k \sim \operatorname{Alt}(p_k),$ where $p_0 \sim R(0,...

**1**

vote

**0**answers

38 views

### Bayesian parameter estimation

I am generally not that knowledgeable for math, so if my question is too broad or inaccurate, please let me know.
I am currently reading a paragraph of one paper (https://www.fil.ion.ucl.ac.uk/spm/...

**0**

votes

**1**answer

131 views

### Shannon problem

Since a few days, I try in my research to model / formalize a source of Shannon a little weird, and I can't do it at all. First of all, I explain to you its operating principle and then I describe it ...

**3**

votes

**1**answer

331 views

### Updating Geman and Geman (1984) on image restoration

I am reading the seminal paper
Stuart Geman and Donald Geman, Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images, IEEE Transactions on Pattern Analysis and Machine ...

**0**

votes

**1**answer

82 views

### How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?

Problem
Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$
Find the marginal distribution of each eigenvalue, using whatever you can.
Background
In my ...

**6**

votes

**0**answers

174 views

### Existence of stick breaking representations for random measures

The Dirichlet process has a roughly size ordered representation in terms of beta random variables, called a stick-breaking representation (Sethuraman, 1994). Similar results hold for the beta process, ...

**2**

votes

**2**answers

289 views

### Quantifying the effect of noise on the posterior variance in Gaussian processes / multivariate Gaussian vectors

Consider a real-valued Gaussian process $f$ on some compact domain $\mathcal{X}$ with mean zero and covariance function $k(x,x') \in [0,1]$ (also known as the kernel function). This question concerns ...

**1**

vote

**0**answers

41 views

### RMHMC sampling in non-parametric setup

The aim is to sample distributions using Fisher information (as mass matrix in Hamiltonian MCMC sampling). Details can be found in http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.190.580&...

**1**

vote

**1**answer

60 views

### A problem with elementary inequality involving probabilities and Brier scoring rule

I am trying to prove certain relations between certain values of the so called Brier inaccuracy measure (Brier scoring rule).
Given a vector $p = (p_1, \ldots p_n)$, where $p_1 + \ldots p_n = 1$ and $...

**0**

votes

**1**answer

77 views

### Learning a Gaussian from noisy observations

Is it possible to learn a distribution over the parameters ($K=\Sigma^{-1}$ and $\mu$) of a Gaussian from noisy measurements of $X$? (Starting with some appropriate prior over the parameters)
I know ...

**3**

votes

**2**answers

329 views

### Multivariate normal concentration

If $X\sim N(0,\Sigma)$ for some $d$-dimensional normal distribution, then $X = \Sigma^{1/2} Z$ where $Z\sim (0,I)$. How to compute the following quantity?
$$
\operatorname{var} (X^T X)
=
\...

**1**

vote

**1**answer

107 views

### Accounting for unobserved events in baysian learning

I wanted to use Bayes theorem to help me automate the task of deciding if I should ignore events, but I am not sure how to update the posterior if I do
The simple story goes like this:
An event $y_i$...

**1**

vote

**1**answer

202 views

### convergence of Bayesian posterior with non iid data

Let $(\epsilon_t)_t$ be a sequence of iid random variables, distributed according to the density $f:\mathbb{R}\to (0,\infty)$ and
$$
x_t = q( \theta^\star, x_1,x_2, \ldots, x_{t-1}) + \epsilon_t \,.
...

**1**

vote

**0**answers

35 views

### Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models.
Consider a hidden Markov model (HMM) with ...

**1**

vote

**1**answer

292 views

### Exploiting conditional independence for inference in Bayesian networks

How is conditional independence used for making probabilistic inference in Bayes networks easier or more efficient?
For example, given the following Bayes network:
Let's say I want to compute ...

**1**

vote

**1**answer

40 views

### Bayesian estimation with lower dimensional prior

Let a statistical model of a random variable $X$ with parameter $\theta \in R^m$ be represented by a density function $p(X=x|\theta)$. Assume that the prior, $q(\cdot)$, is on a lower dimensional ...

**2**

votes

**1**answer

263 views

### Adaptive priors

A lot of recent literature in Bayesian approach to inverse problems involves Adaptive priors, i.e - priors that depend on noise level. A lot of articles deal with optimization of contraction rates ...

**2**

votes

**2**answers

365 views

### Bayes statistics precisely formulated

I am trying to learn something about Bayesian statistics, however, I am struggling already with the simplest equations and, moreover, with the very basic questions: What are we given? What is our goal?...

**8**

votes

**1**answer

299 views

### Base schemes and Bayesian priors

One of Grothendieck's dicta about algebraic geometry is to consider "the relative situation", where one doesn't consider the category of schemes but of schemes over a fixed base scheme.
In Bayesian ...

**2**

votes

**0**answers

52 views

### A canonical example of the non-existence of predictive probability distribution

Section 3 of Fortini et al. (2000) states that
Given $(X^\infty, \mathcal X^\infty,P)$, a predictive probability distribution of $x_n$ given $(x_1, \dots, x_{n-1})$ with respect to $P$ need not ...

**-1**

votes

**1**answer

349 views

### Orthogonal decomposition of conditional expectations

Suppose I have a random variable $x$ and a set of conditional distributions on $x$. Here is an example where the conditionals are nested:
$$q_1 := E(x|y_1), \quad q_2 := E(x|y_1,y_2),\quad q_3 := E(x|...

**7**

votes

**1**answer

924 views

### Rate of convergence of Bayesian posterior

Suppose a data generating process (DGP) is parameterized by some unknown parameter $\theta_0$, say $P_{\theta_0}$, and we want to estimate the value of $\theta_0$ using Bayesian method. Let $\pi(\...