# Questions tagged [linear-regression]

The linear-regression tag has no usage guidance.

**0**

votes

**0**answers

46 views

### How can least squares regression be modified to penalize errors more heavily for small values?

As usual, $f(x_i)$ is some linear combination of the variables, with the error total:
$$\sum_{i=1}^N (y_i - f(x_i))^2$$
I would like to penalize errors near $0$ more heavily. One conceivable ...

**8**

votes

**3**answers

198 views

### Regularized linear vs. RKHS-regression

I'm studying the difference between regularization in RKHS regression and linear regression, but I have a hard time grasping the crucial difference between the two.
Given input-output pairs $(x_i,y_i)...

**0**

votes

**2**answers

66 views

### Unique parameterization of size MxN matrices of rank k

Any rank k matrix $Y\in\mathbb{R}^{m\times n}$ can be written as:
$$ Y = UV'$$
Where $U\in \mathbb{R}^{m\times k}, V\in \mathbb{R}^{n\times k}$. This factorization is not unique since for any ...

**4**

votes

**2**answers

188 views

### Given the joint probability distributions of $X$ and $Y$ for $Y = R\,X+C$, find the probability distributions of $R$ and $C$

Let $R$, $C$, and $X$ be independent random variables defined on $(0,\infty)$ and
$$Y=\underbrace{R\, X}_{Z}+C.$$
We are given the joint probability distribution of $X$ and $Y$, $P_{XY}(x,y)$ and ...

**3**

votes

**1**answer

75 views

### Why to multiply the penalty by $n$ in the penalized least squares and likelihood?

In the SCAD paper by Fan and Li (2001), there exist two forms of penalized least squares as follows:
$$\frac{1}{2}\left \| y-X\beta \right \|^2+\lambda \sum_{j=1}^{d}p_j (\left | \beta _j \right |),$$
...

**2**

votes

**0**answers

35 views

### increasing inter-class distances results in decreasing linear regression error

Let $\{\mathbf{x}_i, y_i \}$ be a set of binary-labeled samples ($\mathbf{x}_i \in \mathbb{R}^d, y_i \in \{a,b\}, a,b\in\mathbb{R}$). Let $\{ \mathbf{x}'_i, y_i \}$ be also such a set.
Define $\mathbf{...

**2**

votes

**1**answer

45 views

### matrix regression under side conditions

I want to solve the folowing problem B*M=V, where B is the unknown of size 3x3, M of size 3xN and V of size 3xN. The difficulty is, that B has to be unitary.
N is in the range of 500. All matrices ...

**1**

vote

**0**answers

77 views

### A different objective function in liner regression analysis

I'm an undergraduate student who is green in statistics. I have a problem in the chose of objective function when estimating the parameters.
Let $Y = \beta^TX + \epsilon $ be the standard liner ...

**0**

votes

**0**answers

135 views

### Solving overdetermined, “polynomial time dependent” linear system

Hello & happy new year,
let $n \geq m$ be two natural numbers. Furthermore let $W \in \mathbb{R}^{n \times m}, Y \in \mathbb{R}^{n \times n}, \lambda \in \mathbb{R}^n, \mu \in \mathbb{R}^m$. ...

**3**

votes

**1**answer

5k views

### Gauss-Newton vs Gradient Descent vs Levenberg-Marquadt for least squared method

I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I now about this topic comes from the book of Taylor "Introduction to error analysis": a set of ...

**1**

vote

**0**answers

56 views

### Posterior consistency of non linear model

This is possibly a reference request. Let $G$ : $\mathbb{R}^p \to \mathbb{R}^q$ be a continuous injective/bijective function. Let $\mu$(we may also assume this to be a non degenerate Gaussian) be ...

**2**

votes

**0**answers

130 views

### Derivation of gradient of SSE in Geodesic Regression

On page 79 (or page 5) of this this paper the gradient of the SSE of the Geodesic model is described explicitly. My question is how are these equitations derived in detail; where can I find the ...

**2**

votes

**3**answers

1k views

### When does a Vandermonde-like matrix have full rank

I have a matrix which is similar to Vandermonde matrix except that the entries are monomials of degree $d$ polynomial in 2 variables. Each row has the following form:
$X_{i}= [1, x_{i}, y_{i}, x_{i}^...

**2**

votes

**0**answers

218 views

### Is there an efficient way to compute the “complete subset regression”?

Background: Let $X \in \mathbb{R}^{N\times K}$ and $y \in \mathbb{R}^{N\times 1}$ be data for a regression problem. The aim is to find $\beta \in \mathbb{R}^{K\times 1}$ such that $X\beta \approx y$ ...

**1**

vote

**1**answer

131 views

### Checking the intersection of two sets

Let $E\subset{\mathbb R}^n$ be a set of the type $I_1\times \dots \times I_n$, where $I_k$ are real intervals, and $X$ be and $n\times p$ real matrix. Suppose also that $rank(X)=p$ and $n>p$. Is ...

**-1**

votes

**1**answer

212 views

### Fitting a quadratic using regression when the y-intercept needs to be 0 [closed]

I'm trying to fit a quadratic $a_0 + a_1x + a_2x^2$ by Polynomial Regression:
$$
\begin{pmatrix}
n & \Sigma x_i & \Sigma x_i\\
\Sigma x_i & \Sigma x_i^2 & \Sigma x_i^3\\
\Sigma ...

**1**

vote

**0**answers

151 views

### How to find all least-square solutions [closed]

I was looking at numpy's lstsq to find a least squares solution of an equation system when the following occurred to me:
Given the points (0,0), (3,4), (4,3), if I ...

**3**

votes

**1**answer

64 views

### Regression with correlation structure

I have a theoretical question about regression models.
Let's say I measured multiple responses from $n$ subjects and these responses are correlated with each other. For example, let's say I measured ...

**1**

vote

**1**answer

2k views

### Minimizing sum of absolute deviations

Suppose we want to find coefficients $b$ in $\underset{b}{\operatorname{argmin}} \displaystyle\sum\limits_{i=1}^n | y_{i}-b_{1}x_{i}-b_{0}\mid$.
If we rewrite this problem in terms of linear ...

**10**

votes

**0**answers

268 views

### Testing contrasts in statistics: Is this provably a hard problem, or not?

Scheffé's method for identifying statistically significant contrasts is widely known. A contrast among the means $\mu_i$, $i=1,\ldots,r$ of $r$ populations is a linear combination $\sum_{i=1}^r c_i \...