# Questions tagged [linear-regression]

The tag has no usage guidance.

31 questions
Filter by
Sorted by
Tagged with
25 views

### Adjust X to strengthen the linearity to Y, in regression model

Assume that we have 2 series X and Y, and obvious we can fit a linear regression model and get all the statistics. I am seeking for some transformation / adjustment which will adjust the value of X, ...
117 views

### Justification of the use of residual plot

$\DeclareMathOperator\Cov{Cov}$Backround of my Question Let $Y$ be the response variable, $\mathbb{X}$ be the explanatory variables. The ultimate goal of prediction is finding a function $f^{*}$ that ...
1 vote
58 views

### Least squares regression with nonnegative error

I'm looking for algorithms to solve a special quadratic programming problem, but I don't know its name or related keywords. Can anyone give me some clues? The problem reads \begin{equation} {\min}_x \...
160 views

1 vote
48 views

### semi-parametric regression

Suppose the observation $(X_1, Y_1), \ldots, (X_n, Y_n)$ satisfies the following semi-parametric model $$Y_t = m(X_t, \alpha) + \sigma(X_t, \beta) U_t,$$ where $U_t$ is independent with $X_t$ with ...
38 views

### The nonparametric estimation in generalized regression model

Let $Y_t \in \mathbb{R}$ be a response variable and $X_t$ a $d$-dimensional explanatory variable. Assume we observe the process that $(X_1, Y_1), \cdots, (X_n, Y_n)$. \begin{equation} Y_{t} = \mu(...
1 vote
459 views

### Hanson-Wright inequality with random matrix

I'm interested in bounding the tail probabilities of a quadratic form $x^t A x$ where $x\in \mathbb{R}^n$ is a sub-Gaussian vector with independent entries. $A\in \mathbb{R}^{n\times n}$ is a matrix. ...
1 vote
85 views

### RKHS/non-parametric regression with missing response values

I am interested in doing RKHS regression with missing response variables. Given input-output pairs $(x_i,y_i)$, I want to estimate a function $f(\cdot)$ as follows \begin{equation}f(x)\approx u(x)=\...
1 vote
114 views

### How to compress variables in a linear regression

I have a large linear regression where all the independent variables are logical (ie TRUE/FALSE) and sparse. The data has roughly 10,000 variables and 10 million observations, on average around 20 ...
1 vote
115 views

305 views

### Unique parameterization of size MxN matrices of rank k

Any rank k matrix $Y\in\mathbb{R}^{m\times n}$ can be written as: $$Y = UV'$$ Where $U\in \mathbb{R}^{m\times k}, V\in \mathbb{R}^{n\times k}$. This factorization is not unique since for any ...
1k views

### Given the joint probability distributions of $X$ and $Y$ for $Y = R\,X+C$, find the probability distributions of $R$ and $C$

Let $R$, $C$, and $X$ be independent random variables defined on $(0,\infty)$ and $$Y=\underbrace{R\, X}_{Z}+C.$$ We are given the joint probability distribution of $X$ and $Y$, $P_{XY}(x,y)$ and ...
114 views

### Why to multiply the penalty by $n$ in the penalized least squares and likelihood?

In the SCAD paper by Fan and Li (2001), there exist two forms of penalized least squares as follows: $$\frac{1}{2}\left \| y-X\beta \right \|^2+\lambda \sum_{j=1}^{d}p_j (\left | \beta _j \right |),$$ ...
46 views

310 views

### Is there an efficient way to compute the "complete subset regression"?

Background: Let $X \in \mathbb{R}^{N\times K}$ and $y \in \mathbb{R}^{N\times 1}$ be data for a regression problem. The aim is to find $\beta \in \mathbb{R}^{K\times 1}$ such that $X\beta \approx y$ ...
1 vote
162 views

### Checking the intersection of two sets

Let $E\subset{\mathbb R}^n$ be a set of the type $I_1\times \dots \times I_n$, where $I_k$ are real intervals, and $X$ be and $n\times p$ real matrix. Suppose also that $rank(X)=p$ and $n>p$. Is ...
611 views

### Fitting a quadratic using regression when the y-intercept needs to be 0 [closed]

I'm trying to fit a quadratic $a_0 + a_1x + a_2x^2$ by Polynomial Regression:  \begin{pmatrix} n & \Sigma x_i & \Sigma x_i\\ \Sigma x_i & \Sigma x_i^2 & \Sigma x_i^3\\ \Sigma ...
1 vote
267 views

### How to find all least-square solutions [closed]

I was looking at numpy's lstsq to find a least squares solution of an equation system when the following occurred to me: Given the points (0,0), (3,4), (4,3), if I ...
210 views

### Regression with correlation structure

I have a theoretical question about regression models. Let's say I measured multiple responses from $n$ subjects and these responses are correlated with each other. For example, let's say I measured ...
1 vote
Suppose we want to find coefficients $b$ in $\underset{b}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n | y_{i}-b_{1}x_{i}-b_{0}\mid$. If we rewrite this problem in terms of linear ...
Scheffé's method for identifying statistically significant contrasts is widely known. A contrast among the means $\mu_i$, $i=1,\ldots,r$ of $r$ populations is a linear combination \$\sum_{i=1}^r c_i \...