Questions tagged [linear-regression]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
27 views

Normalizing a parameter in a regression

I am thinking about the possibility of making a parameter in my regression, let's say the $\lambda$ in a ridge regression, somehow, inside a range : $\lambda \in [0,1]$. Do you have any ideas how I ...
user avatar
0 votes
0 answers
58 views

sharp $l_{\infty}$-bounds for the LASSO estimator

I have a question regarding sharp $l_{\infty}$-bounds for the LASSO estimator. The linear model is $$y=X\theta^*+W,$$ where $X\in\mathbb{R}^{n\times p}$ a deterministic matrix (or a probabilistic one ...
user avatar
1 vote
0 answers
60 views

Fitting a model

I have a function expressed as the ratio of two exponential series with certain parameters $$\frac{\sum\limits_{j=1}^{i-1} \frac{e^{-ar_jt}}{\prod_{l=1\\l \ne j}^{i-1} (b^j-b^l)}}{\sum\limits_{j=1}^{i}...
user avatar
1 vote
0 answers
38 views

semi-parametric regression

Suppose the observation $(X_1, Y_1), \ldots, (X_n, Y_n)$ satisfies the following semi-parametric model $$Y_t = m(X_t, \alpha) + \sigma(X_t, \beta) U_t,$$ where $U_t$ is independent with $X_t$ with ...
user avatar
  • 173
0 votes
1 answer
33 views

The nonparametric estimation in generalized regression model

Let $Y_t \in \mathbb{R}$ be a response variable and $X_t$ a $d$-dimensional explanatory variable. Assume we observe the process that $(X_1, Y_1), \cdots, (X_n, Y_n)$. \begin{equation} Y_{t} = \mu(...
user avatar
  • 173
1 vote
1 answer
368 views

Hanson-Wright inequality with random matrix

I'm interested in bounding the tail probabilities of a quadratic form $x^t A x$ where $x\in \mathbb{R}^n$ is a sub-Gaussian vector with independent entries. $A\in \mathbb{R}^{n\times n}$ is a matrix. ...
user avatar
  • 31
1 vote
3 answers
78 views

RKHS/non-parametric regression with missing response values

I am interested in doing RKHS regression with missing response variables. Given input-output pairs $(x_i,y_i)$, I want to estimate a function $f(\cdot)$ as follows \begin{equation}f(x)\approx u(x)=\...
user avatar
  • 441
1 vote
0 answers
98 views

How to compress variables in a linear regression

I have a large linear regression where all the independent variables are logical (ie TRUE/FALSE) and sparse. The data has roughly 10,000 variables and 10 million observations, on average around 20 ...
user avatar
  • 572
0 votes
0 answers
77 views

Does the intercept converge if we fit a best fit line to points with prime coordinates?

A few months ago I asked this question on Mathematics Stack Exchange but it has received little attention. Perhaps the question is more applicable here. Let $p_k$ denote the $k$th prime such that $...
user avatar
8 votes
3 answers
295 views

Regularized linear vs. RKHS-regression

I'm studying the difference between regularization in RKHS regression and linear regression, but I have a hard time grasping the crucial difference between the two. Given input-output pairs $(x_i,y_i)...
user avatar
  • 441
0 votes
2 answers
206 views

Unique parameterization of size MxN matrices of rank k

Any rank k matrix $Y\in\mathbb{R}^{m\times n}$ can be written as: $$ Y = UV'$$ Where $U\in \mathbb{R}^{m\times k}, V\in \mathbb{R}^{n\times k}$. This factorization is not unique since for any ...
user avatar
4 votes
2 answers
1k views

Given the joint probability distributions of $X$ and $Y$ for $Y = R\,X+C$, find the probability distributions of $R$ and $C$

Let $R$, $C$, and $X$ be independent random variables defined on $(0,\infty)$ and $$Y=\underbrace{R\, X}_{Z}+C.$$ We are given the joint probability distribution of $X$ and $Y$, $P_{XY}(x,y)$ and ...
user avatar
3 votes
1 answer
100 views

Why to multiply the penalty by $n$ in the penalized least squares and likelihood?

In the SCAD paper by Fan and Li (2001), there exist two forms of penalized least squares as follows: $$\frac{1}{2}\left \| y-X\beta \right \|^2+\lambda \sum_{j=1}^{d}p_j (\left | \beta _j \right |),$$ ...
user avatar
  • 31
2 votes
0 answers
43 views

increasing inter-class distances results in decreasing linear regression error

Let $\{\mathbf{x}_i, y_i \}$ be a set of binary-labeled samples ($\mathbf{x}_i \in \mathbb{R}^d, y_i \in \{a,b\}, a,b\in\mathbb{R}$). Let $\{ \mathbf{x}'_i, y_i \}$ be also such a set. Define $\mathbf{...
user avatar
  • 183
2 votes
1 answer
50 views

matrix regression under side conditions

I want to solve the folowing problem B*M=V, where B is the unknown of size 3x3, M of size 3xN and V of size 3xN. The difficulty is, that B has to be unitary. N is in the range of 500. All matrices ...
user avatar
  • 221
1 vote
0 answers
90 views

A different objective function in liner regression analysis

I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters. Let $Y = \beta^TX + \epsilon $ be the standard liner ...
user avatar
  • 11
6 votes
1 answer
16k views

Gauss-Newton vs gradient descent vs Levenberg-Marquadt for least squared method

I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I know about this topic comes from the book of Taylor "Introduction to error analysis": a set ...
user avatar
1 vote
0 answers
62 views

Posterior consistency of non linear model

This is possibly a reference request. Let $G$ : $\mathbb{R}^p \to \mathbb{R}^q$ be a continuous injective/bijective function. Let $\mu$(we may also assume this to be a non degenerate Gaussian) be ...
user avatar
  • 147
2 votes
0 answers
139 views

Derivation of gradient of SSE in Geodesic Regression

On page 79 (or page 5) of this this paper the gradient of the SSE of the Geodesic model is described explicitly. My question is how are these equitations derived in detail; where can I find the ...
user avatar
  • 5,124
3 votes
3 answers
2k views

When does a Vandermonde-like matrix have full rank

I have a matrix which is similar to Vandermonde matrix except that the entries are monomials of degree $d$ polynomial in 2 variables. Each row has the following form: $X_{i}= [1, x_{i}, y_{i}, x_{i}^...
user avatar
  • 133
2 votes
0 answers
303 views

Is there an efficient way to compute the "complete subset regression"?

Background: Let $X \in \mathbb{R}^{N\times K}$ and $y \in \mathbb{R}^{N\times 1}$ be data for a regression problem. The aim is to find $\beta \in \mathbb{R}^{K\times 1}$ such that $X\beta \approx y$ ...
user avatar
  • 326
1 vote
1 answer
148 views

Checking the intersection of two sets

Let $E\subset{\mathbb R}^n$ be a set of the type $I_1\times \dots \times I_n$, where $I_k$ are real intervals, and $X$ be and $n\times p$ real matrix. Suppose also that $rank(X)=p$ and $n>p$. Is ...
user avatar
  • 13
-1 votes
1 answer
518 views

Fitting a quadratic using regression when the y-intercept needs to be 0 [closed]

I'm trying to fit a quadratic $a_0 + a_1x + a_2x^2$ by Polynomial Regression: $$ \begin{pmatrix} n & \Sigma x_i & \Sigma x_i\\ \Sigma x_i & \Sigma x_i^2 & \Sigma x_i^3\\ \Sigma ...
user avatar
  • 101
1 vote
0 answers
238 views

How to find all least-square solutions [closed]

I was looking at numpy's lstsq to find a least squares solution of an equation system when the following occurred to me: Given the points (0,0), (3,4), (4,3), if I ...
user avatar
  • 111
4 votes
1 answer
196 views

Regression with correlation structure

I have a theoretical question about regression models. Let's say I measured multiple responses from $n$ subjects and these responses are correlated with each other. For example, let's say I measured ...
user avatar
1 vote
1 answer
3k views

Minimizing sum of absolute deviations

Suppose we want to find coefficients $b$ in $\underset{b}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n | y_{i}-b_{1}x_{i}-b_{0}\mid$. If we rewrite this problem in terms of linear ...
user avatar
10 votes
0 answers
289 views

Testing contrasts in statistics: Is this provably a hard problem, or not?

Scheffé's method for identifying statistically significant contrasts is widely known. A contrast among the means $\mu_i$, $i=1,\ldots,r$ of $r$ populations is a linear combination $\sum_{i=1}^r c_i \...
user avatar