Questions tagged [bayesian-probability]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
70 views

Derive equation for regularized logistic regression with batch updates

I am trying to understand this paper by Chapelle and Li "An Empirical Evaluation of Thompson Sampling" (2011). In particular, I am failing to derive the equations in algorithm 3 (page 6). ...
user avatar
2 votes
0 answers
35 views

Concentration of posterior probability around a tiny fraction of the prior volume

In the context of approximating the evidence $Z$ in a Bayesian inference setting $$ Z = \int d\theta \mathcal L (\theta)\pi (\theta) $$ with $\mathcal L$ the likelihood, $\pi$ the prior, John Skilling'...
user avatar
1 vote
1 answer
80 views

Bayesian inverse problems on non-separable Banach spaces

I am now studying Bayesian inverse problems. In the note of Dashti and Stuart https://arxiv.org/abs/1302.6989, they mentioned that "... when considering a non-separable Banach space $B$, it is ...
user avatar
1 vote
1 answer
70 views

Conditional Gaussians in infinite dimensions

I asked this over on cross validated, but thought it might also get an answer here: The law of the conditional Gaussian distribution (the mean and covariance) are frequently mentioned to extend to the ...
user avatar
0 votes
0 answers
39 views

Bayesian hierarchical model inference - real problem

it might be really confusing question. I am working on my thesis and I am stuck at a problem. It's a problem in image segmentation and finding parameters of border lines of continuous region in an ...
user avatar
  • 11
4 votes
2 answers
200 views

Do these distributions have a name already?

In playing with some math finance stuff I ran into the following distribution and I was curious if someone had a name for it or has studied it or worked with it already. To start, let $\Delta^n$ be ...
user avatar
3 votes
1 answer
245 views

A quantity associated to a probability measure space

Let $(S,P)$ be a (finite) probability space. We associate to $(S,P)$ a quantity $n(S,P)$ as follows: The probability of two randomly chosen events $A,B\subset S$ being independent is denoted by $n(S,P)...
user avatar
0 votes
0 answers
84 views

KL divergence between two sequences

Let us have a random sequence $(X_1, Y_1,\ldots,X_n,Y_n)$, where $X_t$ takes value in some set $\mathcal{X}$ and $Y_i$ are scalars. The sequence is generated by the following process: $X_i$ is chosen ...
user avatar
  • 173
3 votes
1 answer
176 views

Gaussian process kernel parameter tuning

I am reading on gaussian processes and there are multiple resources that say how the parameters of the prior (kernel, mean) can be fitted based on data,specifically by choosing those that maximize the ...
user avatar
  • 131
1 vote
0 answers
24 views

Estimation of probability matrix from samples at different time intervals

I am given discrete-time Markov chain that evolves on a finite subset $\{1,\dots,n\}$. This Markov chain is time-homogeneous and has a transition matrix $P$ that I want to estimate. Let $X_t$ be the ...
user avatar
  • 542
0 votes
1 answer
113 views

CLT for random variables with positive support (e.g. exponential)

I have a bunch of iid $\{X_i\}$ with $X_i \sim \exp(\lambda)$ - let's say $\lambda = 1$. Now, classic version of CLT tells me: \begin{equation} \sqrt{n}\left(1-\bar{X}_n\right) \rightarrow \mathcal{N}\...
user avatar
  • 91
0 votes
0 answers
40 views

2d interpolation minimizing the integral of the norm of the Hessian

It is well known that cubic interpolation is the solution of the interpolation problem that minimizes the integral of the square of the second derivative: $$ min_{f \text{ s.t. } f(x_i)=y_i} \int (f''(...
user avatar
0 votes
1 answer
79 views

Lower bound for reduced variance after conditioning

Let $X$ be a random variable with variance $\tau^2$ and $Y$ be another random variable such that $Y-X$ is independent of $X$ and has mean zero and variance $\sigma^2$. (One can think of $Y$ as a noisy ...
user avatar
  • 3
-1 votes
1 answer
32 views

Linear operator over a simplex space in a multinomial distribution parameter estimation problem

This is actually a variant of a well-known problem of how the parameters of a multinomial distribution can be estimated by maximum likelihood, and this arises from a final year project I undertook ...
user avatar
1 vote
1 answer
212 views

Posterior expected value for squared Fourier coefficients of random Boolean function

Let $f : \{0, 1\}^{n} \rightarrow \{-1, 1\}$ be a Boolean function. Let the Fourier coefficients of this function be given by $$ \hat f(z) = \frac{1}{2^{n}} \sum_{x \in \{0, 1\}^{n}} f(x)(-1)^{x \cdot ...
user avatar
1 vote
0 answers
65 views

Bayesian inference of stochastically evolving model parameters

I have a question related to self-calibration in radio interferometry, but I will try to phrase it as generic as possible. I have a set of data points, $D = \{ d_{0, t_0}, d_{1, t_0}, ..., d_{M, t_0}, ...
user avatar
  • 29
1 vote
1 answer
571 views

Convolution of two Gaussian mixture model

Suppose I have two independent random variables $X$, $Y$, each modeled by the Gaussian mixture model (GMM). That is, $$ f(x)=\sum _{k=1}^K \pi _k \mathcal{N}\left(x|\mu _k,\sigma _k\right) $$ $$ g(y)=\...
user avatar
3 votes
0 answers
132 views

Minimizing an f-divergence and Jeffrey's Rule

My question is about f-divergences and Richard Jeffrey's (1965) rule for updating probabilities in the light of partial information. The set-up: Let $p: \mathcal{F} \rightarrow [0,1]$ be a ...
user avatar
  • 91
1 vote
0 answers
37 views

Quantitative bounds on convergence of Bayesian posterior

Let $Y$ be a random variable in $[0,1]$, and let $X_1, X_2, \ldots$ be a sequence of random variables in $[0,1]$. Suppose that the $X_i$'s are conditionally i.i.d given $Y$ ; in other words, I'd like ...
user avatar
  • 173
2 votes
0 answers
87 views

Is there any good reference on the Bayesian view that can be helpful for reading papers on the number theory using heuristic arguments?

Nowadays there are many papers on the number theory using heuristics. I have read some of them. But I have no clear understanding of the Bayesian Probability(subjective probability). The concept of ...
user avatar
  • 951
4 votes
0 answers
144 views

Convergence of the expectation of a random variable when conditioned on its sum with another, independent but not identically distributed

Suppose that for all $n \in \mathbf{N}$, $X_n$ and $Y_n$ are independent random variables with $$X_n \sim \mathtt{Binomial}(n,1-q),$$ and $$Y_n \sim \mathtt{Poisson}(n(q+\epsilon_n)),$$ where $q \in (...
user avatar
  • 91
-1 votes
1 answer
152 views

Proving the existence of a symmetric Bayesian Nash equilibrium

I am currently faced with the following question: Consider the public goods game. Suppose that there are $I > 2$ players and that the public goods is supplied (with benefit of 1 for all players) ...
user avatar
0 votes
0 answers
37 views

restriction of a formula with matrix inverse multiplied by a vector

I'm trying to reproduce a proof from this paper but I'm stuck in one point (Lemma 6). The general subject is bayesian model for multi-armed bandit problem solved with Thompson sampling. I think I ...
user avatar
1 vote
0 answers
36 views

Bayesian posterior consistency when prior distribution is induced by a diffusion

Let $\Pi_{b,\sigma}$ be a prior distribution on $\{z_t\}_{t<T}\in C_0[0,T]$ induced by the following diffusion: \begin{align} d\tilde z_t&=b(\tilde z_t,t)dt+\sigma(\tilde z_t,t) dW_t, ~...
user avatar
1 vote
0 answers
70 views

Convergence of Bayesian posterior

Let $\Delta [0,1]$ denote the set of all probability distributions on the unit interval. Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density ...
user avatar
1 vote
1 answer
89 views

Conditional density for random effects prediction in GLMM

I am currently working on generalized linear mixed models (GLMM) and need some help concerning the prediction of the random effects. More specifically, I don't understand the given representation of ...
user avatar
  • 13
0 votes
1 answer
302 views

Optimal solution to cross entropy loss in the continuous case

This could be a simple question but I don't have a satisfying answer. Setup. Suppose that we have $K$ different classes, and consider cross entropy loss which maps a probability vector in the ...
user avatar
  • 143
9 votes
3 answers
317 views

What does the KL being symmetric tell us about the distributions?

Suppose two probability density functions, $p$ and $q$, such that $\text{KL}(q||p) = \text{KL}(p||q) \neq 0$. Intuitively, does that tell us anything interesting about the nature of these densities?
user avatar
  • 113
3 votes
0 answers
79 views

Have stick-breaking priors with non-iid atoms been considered, and if not, why not?

Roughly speaking, a stick-breaking prior is a random discrete probability measure $P$ on a measurable space $\mathcal X$ of the form $$P=\sum_{j\ge1}w_j\delta_{\theta_j}$$ where $(w_j)_{j\ge1}$ is a ...
user avatar
3 votes
1 answer
1k views

Bayesian Inference with Student-t likelihood

Suppose I've observed $x$ from a Student-t distribution with unknown $\mu$, and I'd now like to infer $\mu$. Since the t-distribution isn't exponential family, there's no conjugate prior available, ...
user avatar
  • 41
5 votes
1 answer
247 views

Bounding the sensitivity of a posterior mean to changes in a single data point

There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...
user avatar
  • 1,058
4 votes
0 answers
440 views

Bayesian Networks and Polytree

I am a bit puzzled by the use of polytree to infer a posterior in a Bayesian Network (BN). BN are defined as directed acyclic graphs. A polytree is DAG whose underlying undirected graph is a tree. ...
user avatar
  • 41
2 votes
2 answers
381 views

Parametrising a sparse orthogonal matrix

I need to find a way to parametrise a matrix that is both sparse (to some degree) and orthogonal, i.e., I am looking for a parametrisation that describes $A \in \mathbb{R}^{n\times m}$ such that $AA^𝑇...
user avatar
  • 113
1 vote
0 answers
269 views

Gaussian Integrals over Spheres

I'm after a reference for an integral. In particular, I am looking a way to approximate or calculate the following: $$ \int \limits_{\| \theta \|_2 = 1} e^{(-(\theta - \mu)^T \Sigma (\theta - \mu))} ...
user avatar
1 vote
1 answer
96 views

The expectation of binary logistics regression with respect to Gaussian distribution

I am trying to compute the expectation of $g(s,x)=s \ln \sigma(x)+(1-s)\ln(1-\sigma(x))$ with respect to the normal distribution $\mathcal{N}(x;m,v)$, where we have $\sigma(x)=\frac{1}{1+e^{-x}}$. If ...
user avatar
  • 29
1 vote
1 answer
192 views

Bayesian methods in online setting

Imagine the following (very concrete) model: We have a series of random variables $x_k$ with values in $\lbrace 0, 1\rbrace$. We assume $x_k \mid p_k \sim \operatorname{Alt}(p_k),$ where $p_0 \sim R(0,...
user avatar
  • 141
1 vote
0 answers
47 views

Bayesian parameter estimation

I am generally not that knowledgeable for math, so if my question is too broad or inaccurate, please let me know. I am currently reading a paragraph of one paper (https://www.fil.ion.ucl.ac.uk/spm/...
user avatar
0 votes
1 answer
137 views

Shannon problem

Since a few days, I try in my research to model / formalize a source of Shannon a little weird, and I can't do it at all. First of all, I explain to you its operating principle and then I describe it ...
user avatar
3 votes
1 answer
366 views

Updating Geman and Geman (1984) on image restoration

I am reading the seminal paper Stuart Geman and Donald Geman, Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images, IEEE Transactions on Pattern Analysis and Machine ...
user avatar
  • 75
0 votes
1 answer
85 views

How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?

Problem Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$ Find the marginal distribution of each eigenvalue, using whatever you can. Background In my ...
user avatar
6 votes
0 answers
176 views

Existence of stick breaking representations for random measures

The Dirichlet process has a roughly size ordered representation in terms of beta random variables, called a stick-breaking representation (Sethuraman, 1994). Similar results hold for the beta process, ...
user avatar
2 votes
2 answers
389 views

Quantifying the effect of noise on the posterior variance in Gaussian processes / multivariate Gaussian vectors

Consider a real-valued Gaussian process $f$ on some compact domain $\mathcal{X}$ with mean zero and covariance function $k(x,x') \in [0,1]$ (also known as the kernel function). This question concerns ...
user avatar
1 vote
0 answers
42 views

RMHMC sampling in non-parametric setup

The aim is to sample distributions using Fisher information (as mass matrix in Hamiltonian MCMC sampling). Details can be found in http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.190.580&...
user avatar
  • 147
1 vote
1 answer
70 views

A problem with elementary inequality involving probabilities and Brier scoring rule

I am trying to prove certain relations between certain values of the so called Brier inaccuracy measure (Brier scoring rule). Given a vector $p = (p_1, \ldots p_n)$, where $p_1 + \ldots p_n = 1$ and $...
user avatar
  • 135
0 votes
1 answer
94 views

Learning a Gaussian from noisy observations

Is it possible to learn a distribution over the parameters ($K=\Sigma^{-1}$ and $\mu$) of a Gaussian from noisy measurements of $X$? (Starting with some appropriate prior over the parameters) I know ...
user avatar
  • 21
3 votes
2 answers
365 views

Multivariate normal concentration

If $X\sim N(0,\Sigma)$ for some $d$-dimensional normal distribution, then $X = \Sigma^{1/2} Z$ where $Z\sim (0,I)$. How to compute the following quantity? $$ \operatorname{var} (X^T X) = \...
user avatar
1 vote
1 answer
110 views

Accounting for unobserved events in baysian learning

I wanted to use Bayes theorem to help me automate the task of deciding if I should ignore events, but I am not sure how to update the posterior if I do The simple story goes like this: An event $y_i$...
user avatar
  • 213
1 vote
1 answer
238 views

convergence of Bayesian posterior with non iid data

Let $(\epsilon_t)_t$ be a sequence of iid random variables, distributed according to the density $f:\mathbb{R}\to (0,\infty)$ and $$ x_t = q( \theta^\star, x_1,x_2, \ldots, x_{t-1}) + \epsilon_t \,. ...
user avatar
  • 355
1 vote
0 answers
41 views

Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
user avatar
  • 171
1 vote
1 answer
468 views

Exploiting conditional independence for inference in Bayesian networks

How is conditional independence used for making probabilistic inference in Bayes networks easier or more efficient? For example, given the following Bayes network: Let's say I want to compute ...
user avatar