# Questions tagged [linear-regression]

The tag has no usage guidance.

25 questions
Filter by
Sorted by
Tagged with
29 views

### semi-parametric regression

Suppose the observation $(X_1, Y_1), \ldots, (X_n, Y_n)$ satisfies the following semi-parametric model $$Y_t = m(X_t, \alpha) + \sigma(X_t, \beta) U_t,$$ where $U_t$ is independent with $X_t$ with ...
46 views

### Linear regression with a small twist, interesting and in-depth

We have a matrix $A \in \mathbb R^{n \times d}$ which we try and fit to a vector $b \in \mathbb R^{n \times 1}$ in the following way: We attempt to find a matrix $W \in \mathbb R^{d \times d}$ and a ...
29 views

### The nonparametric estimation in generalized regression model

Let $Y_t \in \mathbb{R}$ be a response variable and $X_t$ a $d$-dimensional explanatory variable. Assume we observe the process that $(X_1, Y_1), \cdots, (X_n, Y_n)$. \begin{equation} Y_{t} = \mu(...
190 views

### Hanson-Wright inequality with random matrix

I'm interested in bounding the tail probabilities of a quadratic form $x^t A x$ where $x\in \mathbb{R}^n$ is a sub-Gaussian vector with independent entries. $A\in \mathbb{R}^{n\times n}$ is a matrix. ...
71 views

### RKHS/non-parametric regression with missing response values

I am interested in doing RKHS regression with missing response variables. Given input-output pairs $(x_i,y_i)$, I want to estimate a function $f(\cdot)$ as follows \begin{equation}f(x)\approx u(x)=\...
76 views

### How to compress variables in a linear regression

I have a large linear regression where all the independent variables are logical (ie TRUE/FALSE) and sparse. The data has roughly 10,000 variables and 10 million observations, on average around 20 ...
77 views

126 views

### Unique parameterization of size MxN matrices of rank k

Any rank k matrix $Y\in\mathbb{R}^{m\times n}$ can be written as: $$Y = UV'$$ Where $U\in \mathbb{R}^{m\times k}, V\in \mathbb{R}^{n\times k}$. This factorization is not unique since for any ...
671 views

### Given the joint probability distributions of $X$ and $Y$ for $Y = R\,X+C$, find the probability distributions of $R$ and $C$

Let $R$, $C$, and $X$ be independent random variables defined on $(0,\infty)$ and $$Y=\underbrace{R\, X}_{Z}+C.$$ We are given the joint probability distribution of $X$ and $Y$, $P_{XY}(x,y)$ and ...
86 views

### Why to multiply the penalty by $n$ in the penalized least squares and likelihood?

In the SCAD paper by Fan and Li (2001), there exist two forms of penalized least squares as follows: $$\frac{1}{2}\left \| y-X\beta \right \|^2+\lambda \sum_{j=1}^{d}p_j (\left | \beta _j \right |),$$ ...
43 views

294 views

### Is there an efficient way to compute the “complete subset regression”?

Background: Let $X \in \mathbb{R}^{N\times K}$ and $y \in \mathbb{R}^{N\times 1}$ be data for a regression problem. The aim is to find $\beta \in \mathbb{R}^{K\times 1}$ such that $X\beta \approx y$ ...
136 views

### Checking the intersection of two sets

Let $E\subset{\mathbb R}^n$ be a set of the type $I_1\times \dots \times I_n$, where $I_k$ are real intervals, and $X$ be and $n\times p$ real matrix. Suppose also that $rank(X)=p$ and $n>p$. Is ...
428 views

### Fitting a quadratic using regression when the y-intercept needs to be 0 [closed]

I'm trying to fit a quadratic $a_0 + a_1x + a_2x^2$ by Polynomial Regression:  \begin{pmatrix} n & \Sigma x_i & \Sigma x_i\\ \Sigma x_i & \Sigma x_i^2 & \Sigma x_i^3\\ \Sigma ...
216 views

### How to find all least-square solutions [closed]

I was looking at numpy's lstsq to find a least squares solution of an equation system when the following occurred to me: Given the points (0,0), (3,4), (4,3), if I ...
178 views

### Regression with correlation structure

I have a theoretical question about regression models. Let's say I measured multiple responses from $n$ subjects and these responses are correlated with each other. For example, let's say I measured ...
Suppose we want to find coefficients $b$ in $\underset{b}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n | y_{i}-b_{1}x_{i}-b_{0}\mid$. If we rewrite this problem in terms of linear ...
Scheffé's method for identifying statistically significant contrasts is widely known. A contrast among the means $\mu_i$, $i=1,\ldots,r$ of $r$ populations is a linear combination \$\sum_{i=1}^r c_i \...