Questions tagged [bayesian-probability]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
3
votes
1answer
59 views

Gaussian process kernel parameter tuning

I am reading on gaussian processes and there are multiple resources that say how the parameters of the prior (kernel, mean) can be fitted based on data,specifically by choosing those that maximize the ...
0
votes
0answers
19 views

Estimation of probability matrix from samples at different time intervals

I am given discrete-time Markov chain that evolves on a finite subset $\{1,\dots,n\}$. This Markov chain is time-homogeneous and has a transition matrix $P$ that I want to estimate. Let $X_t$ be the ...
0
votes
0answers
25 views

Integrating multinomial likelihood over intersection of two simplexes

I am trying to calculate this integral, coming from a multinomial likelihood with an extra condition ($\sum_{i=1}^n p_i w_i = 1$). Essentially it can be seen as integrating a Dirichlet pdf over a ...
0
votes
1answer
93 views

CLT for random variables with positive support (e.g. exponential)

I have a bunch of iid $\{X_i\}$ with $X_i \sim \exp(\lambda)$ - let's say $\lambda = 1$. Now, classic version of CLT tells me: \begin{equation} \sqrt{n}\left(1-\bar{X}_n\right) \rightarrow \mathcal{N}\...
0
votes
0answers
34 views

2d interpolation minimizing the integral of the norm of the Hessian

It is well known that cubic interpolation is the solution of the interpolation problem that minimizes the integral of the square of the second derivative: $$ min_{f \text{ s.t. } f(x_i)=y_i} \int (f''(...
0
votes
1answer
55 views

Lower bound for reduced variance after conditioning

Let $X$ be a random variable with variance $\tau^2$ and $Y$ be another random variable such that $Y-X$ is independent of $X$ and has mean zero and variance $\sigma^2$. (One can think of $Y$ as a noisy ...
-1
votes
1answer
27 views

Linear operator over a simplex space in a multinomial distribution parameter estimation problem

This is actually a variant of a well-known problem of how the parameters of a multinomial distribution can be estimated by maximum likelihood, and this arises from a final year project I undertook ...
1
vote
1answer
155 views

Posterior expected value for squared Fourier coefficients of random Boolean function

Let $f : \{0, 1\}^{n} \rightarrow \{-1, 1\}$ be a Boolean function. Let the Fourier coefficients of this function be given by $$ \hat f(z) = \frac{1}{2^{n}} \sum_{x \in \{0, 1\}^{n}} f(x)(-1)^{x \cdot ...
1
vote
0answers
53 views

Bayesian inference of stochastically evolving model parameters

I have a question related to self-calibration in radio interferometry, but I will try to phrase it as generic as possible. I have a set of data points, $D = \{ d_{0, t_0}, d_{1, t_0}, ..., d_{M, t_0}, ...
1
vote
1answer
195 views

Convolution of two Gaussian mixture model

Suppose I have two independent random variables $X$, $Y$, each modeled by the Gaussian mixture model (GMM). That is, $$ f(x)=\sum _{k=1}^K \pi _k \mathcal{N}\left(x|\mu _k,\sigma _k\right) $$ $$ g(y)=\...
2
votes
0answers
108 views

Minimizing an f-divergence and Jeffrey's Rule

My question is about f-divergences and Richard Jeffrey's (1965) rule for updating probabilities in the light of partial information. The set-up: Let $p: \mathcal{F} \rightarrow [0,1]$ be a ...
1
vote
0answers
36 views

Quantitative bounds on convergence of Bayesian posterior

Let $Y$ be a random variable in $[0,1]$, and let $X_1, X_2, \ldots$ be a sequence of random variables in $[0,1]$. Suppose that the $X_i$'s are conditionally i.i.d given $Y$ ; in other words, I'd like ...
2
votes
0answers
82 views

Is there any good reference on the Bayesian view that can be helpful for reading papers on the number theory using heuristic arguments?

Nowadays there are many papers on the number theory using heuristics. I have read some of them. But I have no clear understanding of the Bayesian Probability(subjective probability). The concept of ...
4
votes
0answers
107 views

Convergence of the expectation of a random variable when conditioned on its sum with another, independent but not identically distributed

Suppose that for all $n \in \mathbf{N}$, $X_n$ and $Y_n$ are independent random variables with $$X_n \sim \mathtt{Binomial}(n,1-q),$$ and $$Y_n \sim \mathtt{Poisson}(n(q+\epsilon_n)),$$ where $q \in (...
0
votes
1answer
89 views

Proving the existence of a symmetric Bayesian Nash equilibrium

I am currently faced with the following question: Consider the public goods game. Suppose that there are $I > 2$ players and that the public goods is supplied (with benefit of 1 for all players) ...
0
votes
0answers
33 views

restriction of a formula with matrix inverse multiplied by a vector

I'm trying to reproduce a proof from this paper but I'm stuck in one point (Lemma 6). The general subject is bayesian model for multi-armed bandit problem solved with Thompson sampling. I think I ...
1
vote
0answers
32 views

Bayesian posterior consistency when prior distribution is induced by a diffusion

Let $\Pi_{b,\sigma}$ be a prior distribution on $\{z_t\}_{t<T}\in C_0[0,T]$ induced by the following diffusion: \begin{align} d\tilde z_t&=b(\tilde z_t,t)dt+\sigma(\tilde z_t,t) dW_t, ~...
1
vote
0answers
68 views

Convergence of Bayesian posterior

Let $\Delta [0,1]$ denote the set of all probability distributions on the unit interval. Let $\mu \in \Delta [0,1]$ denote an arbitrary prior. Importantly, $\mu$ does not necessarily admit a density ...
1
vote
1answer
61 views

Conditional density for random effects prediction in GLMM

I am currently working on generalized linear mixed models (GLMM) and need some help concerning the prediction of the random effects. More specifically, I don't understand the given representation of ...
0
votes
1answer
206 views

Optimal solution to cross entropy loss in the continuous case

This could be a simple question but I don't have a satisfying answer. Setup. Suppose that we have $K$ different classes, and consider cross entropy loss which maps a probability vector in the ...
8
votes
3answers
242 views

What does the KL being symmetric tell us about the distributions?

Suppose two probability density functions, $p$ and $q$, such that $\text{KL}(q||p) = \text{KL}(p||q) \neq 0$. Intuitively, does that tell us anything interesting about the nature of these densities?
3
votes
0answers
43 views

Have stick-breaking priors with non-iid atoms been considered, and if not, why not?

Roughly speaking, a stick-breaking prior is a random discrete probability measure $P$ on a measurable space $\mathcal X$ of the form $$P=\sum_{j\ge1}w_j\delta_{\theta_j}$$ where $(w_j)_{j\ge1}$ is a ...
3
votes
1answer
576 views

Bayesian Inference with Student-t likelihood

Suppose I've observed $x$ from a Student-t distribution with unknown $\mu$, and I'd now like to infer $\mu$. Since the t-distribution isn't exponential family, there's no conjugate prior available, ...
5
votes
1answer
209 views

Bounding the sensitivity of a posterior mean to changes in a single data point

There is a real-valued random variable $R$. Define a finite set of random variables ("data points") $$X_i = R + Z_i \; \text{for } i\in\{1,\ldots,n\},$$ where $Z_i$ are identically and independently ...
4
votes
0answers
295 views

Bayesian Networks and Polytree

I am a bit puzzled by the use of polytree to infer a posterior in a Bayesian Network (BN). BN are defined as directed acyclic graphs. A polytree is DAG whose underlying undirected graph is a tree. ...
2
votes
2answers
254 views

Parametrising a sparse orthogonal matrix

I need to find a way to parametrise a matrix that is both sparse (to some degree) and orthogonal, i.e., I am looking for a parametrisation that describes $A \in \mathbb{R}^{n\times m}$ such that $AA^𝑇...
1
vote
0answers
177 views

Gaussian Integrals over Spheres

I'm after a reference for an integral. In particular, I am looking a way to approximate or calculate the following: $$ \int \limits_{\| \theta \|_2 = 1} e^{(-(\theta - \mu)^T \Sigma (\theta - \mu))} ...
1
vote
1answer
92 views

The expectation of binary logistics regression with respect to Gaussian distribution

I am trying to compute the expectation of $g(s,x)=s \ln \sigma(x)+(1-s)\ln(1-\sigma(x))$ with respect to the normal distribution $\mathcal{N}(x;m,v)$, where we have $\sigma(x)=\frac{1}{1+e^{-x}}$. If ...
1
vote
1answer
143 views

Bayesian methods in online setting

Imagine the following (very concrete) model: We have a series of random variables $x_k$ with values in $\lbrace 0, 1\rbrace$. We assume $x_k \mid p_k \sim \operatorname{Alt}(p_k),$ where $p_0 \sim R(0,...
1
vote
0answers
38 views

Bayesian parameter estimation

I am generally not that knowledgeable for math, so if my question is too broad or inaccurate, please let me know. I am currently reading a paragraph of one paper (https://www.fil.ion.ucl.ac.uk/spm/...
0
votes
1answer
131 views

Shannon problem

Since a few days, I try in my research to model / formalize a source of Shannon a little weird, and I can't do it at all. First of all, I explain to you its operating principle and then I describe it ...
3
votes
1answer
331 views

Updating Geman and Geman (1984) on image restoration

I am reading the seminal paper Stuart Geman and Donald Geman, Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images, IEEE Transactions on Pattern Analysis and Machine ...
0
votes
1answer
82 views

How to infer the eigenvalue distribution from matrix where each entry has a known Gaussian distribution?

Problem Given $X \in \mathbb{R}^{n \times n}$ where $X_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma_{ij}^2 I)$ Find the marginal distribution of each eigenvalue, using whatever you can. Background In my ...
6
votes
0answers
174 views

Existence of stick breaking representations for random measures

The Dirichlet process has a roughly size ordered representation in terms of beta random variables, called a stick-breaking representation (Sethuraman, 1994). Similar results hold for the beta process, ...
2
votes
2answers
289 views

Quantifying the effect of noise on the posterior variance in Gaussian processes / multivariate Gaussian vectors

Consider a real-valued Gaussian process $f$ on some compact domain $\mathcal{X}$ with mean zero and covariance function $k(x,x') \in [0,1]$ (also known as the kernel function). This question concerns ...
1
vote
0answers
41 views

RMHMC sampling in non-parametric setup

The aim is to sample distributions using Fisher information (as mass matrix in Hamiltonian MCMC sampling). Details can be found in http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.190.580&...
1
vote
1answer
60 views

A problem with elementary inequality involving probabilities and Brier scoring rule

I am trying to prove certain relations between certain values of the so called Brier inaccuracy measure (Brier scoring rule). Given a vector $p = (p_1, \ldots p_n)$, where $p_1 + \ldots p_n = 1$ and $...
0
votes
1answer
77 views

Learning a Gaussian from noisy observations

Is it possible to learn a distribution over the parameters ($K=\Sigma^{-1}$ and $\mu$) of a Gaussian from noisy measurements of $X$? (Starting with some appropriate prior over the parameters) I know ...
3
votes
2answers
329 views

Multivariate normal concentration

If $X\sim N(0,\Sigma)$ for some $d$-dimensional normal distribution, then $X = \Sigma^{1/2} Z$ where $Z\sim (0,I)$. How to compute the following quantity? $$ \operatorname{var} (X^T X) = \...
1
vote
1answer
107 views

Accounting for unobserved events in baysian learning

I wanted to use Bayes theorem to help me automate the task of deciding if I should ignore events, but I am not sure how to update the posterior if I do The simple story goes like this: An event $y_i$...
1
vote
1answer
202 views

convergence of Bayesian posterior with non iid data

Let $(\epsilon_t)_t$ be a sequence of iid random variables, distributed according to the density $f:\mathbb{R}\to (0,\infty)$ and $$ x_t = q( \theta^\star, x_1,x_2, \ldots, x_{t-1}) + \epsilon_t \,. ...
1
vote
0answers
35 views

Bounding Hidden Markov model Bayesian filter error with inexact models

In context of a hidden Markov model, I am interested in bounding the error of a Bayesian filter when using inexact state transition and observation models. Consider a hidden Markov model (HMM) with ...
1
vote
1answer
292 views

Exploiting conditional independence for inference in Bayesian networks

How is conditional independence used for making probabilistic inference in Bayes networks easier or more efficient? For example, given the following Bayes network: Let's say I want to compute ...
1
vote
1answer
40 views

Bayesian estimation with lower dimensional prior

Let a statistical model of a random variable $X$ with parameter $\theta \in R^m$ be represented by a density function $p(X=x|\theta)$. Assume that the prior, $q(\cdot)$, is on a lower dimensional ...
2
votes
1answer
263 views

Adaptive priors

A lot of recent literature in Bayesian approach to inverse problems involves Adaptive priors, i.e - priors that depend on noise level. A lot of articles deal with optimization of contraction rates ...
2
votes
2answers
365 views

Bayes statistics precisely formulated

I am trying to learn something about Bayesian statistics, however, I am struggling already with the simplest equations and, moreover, with the very basic questions: What are we given? What is our goal?...
8
votes
1answer
299 views

Base schemes and Bayesian priors

One of Grothendieck's dicta about algebraic geometry is to consider "the relative situation", where one doesn't consider the category of schemes but of schemes over a fixed base scheme. In Bayesian ...
2
votes
0answers
52 views

A canonical example of the non-existence of predictive probability distribution

Section 3 of Fortini et al. (2000) states that Given $(X^\infty, \mathcal X^\infty,P)$, a predictive probability distribution of $x_n$ given $(x_1, \dots, x_{n-1})$ with respect to $P$ need not ...
-1
votes
1answer
349 views

Orthogonal decomposition of conditional expectations

Suppose I have a random variable $x$ and a set of conditional distributions on $x$. Here is an example where the conditionals are nested: $$q_1 := E(x|y_1), \quad q_2 := E(x|y_1,y_2),\quad q_3 := E(x|...
7
votes
1answer
924 views

Rate of convergence of Bayesian posterior

Suppose a data generating process (DGP) is parameterized by some unknown parameter $\theta_0$, say $P_{\theta_0}$, and we want to estimate the value of $\theta_0$ using Bayesian method. Let $\pi(\...