# Questions tagged [least-squares]

The least-squares tag has no usage guidance.

26
questions

**0**

votes

**0**answers

23 views

### Nonlinear least-squares parameter estimation of a vector-valued function

I'm given a function $f\, (x; \vec{\beta})$, non-linear, where ${x} \in \mathbb{R}$ and $\vec{\beta} \in \mathbb{R}^k$. It models an experiment of a setup characterized by the parameters $\vec{\beta}$....

**19**

votes

**4**answers

999 views

### Is the pseudoinverse the same as least squares with regularization?

Given a linear system $Ax=b$, the pseudoinverse of $A$ is found as the matrix $A^+$ such that $x=A^+ b$ where $x$ solves the least squares problem $\min \| Ax - b \|^2 $ and $x \perp \mathcal{N}(A)$. ...

**2**

votes

**1**answer

142 views

### Concentration inequality for norm of solution to nonlinear least-squares problem

Define the piecewise-linear function $\psi(t):=\max(t,0)$ for all $t \in \mathbb R$.
Let $d,n,k \to \infty$ at the same rate (i.e $n \asymp k \asymp d$).
Let $y_1,\ldots,y_n \in \{-1,1\}$ uniformly ...

**1**

vote

**2**answers

114 views

### Robust estimation of $Ax=b$

Problem setting :
$ \underset{x}{\text{min}} \|Ax-b\|$, where $A \in \mathcal{R}^{m \times n}, m\gg n $, full rank.
L1 loss is used for robust estimation using IRLS. The corresponding equation to ...

**1**

vote

**1**answer

76 views

### Understanding the Time Delay of Arrival trilateration algorithm

I'm trying to algorithmically solve the Time Delay of Arrival problem as part of some mathematics research. The problem is as follows:
Given the location of three receivers in a plane (A, B, and C), ...

**0**

votes

**0**answers

41 views

### unusual least square solution in SINDy algorithm

In the SINDy algorithm, the equation $\dot{X} \approx \Theta\Xi$ has to be solved for $\Xi$. $\dot{X}$ and $\Theta$ are known numerical matricies. The intuitive solution to this problem would be to ...

**1**

vote

**2**answers

63 views

### Non-parametric regression and curvature

Given a finite set of points $(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)$ in the plane, Linear Regression tells us how to find the straight line "$y=a+bx$" best approximating the given points, in the ...

**1**

vote

**3**answers

70 views

### RKHS/non-parametric regression with missing response values

I am interested in doing RKHS regression with missing response variables.
Given input-output pairs $(x_i,y_i)$, I want to estimate a function $f(\cdot)$ as follows
\begin{equation}f(x)\approx u(x)=\...

**2**

votes

**1**answer

288 views

### Minimise $\sum_i \begin{Vmatrix}\boldsymbol{x}_i \\ \boldsymbol{y}_i\end{Vmatrix}$

Consider column vectors $\boldsymbol{z}_i$, $\quad i=1,\dots,n$.
Each $\boldsymbol{z}_i$ has $j$ elements and can be expressed as $\boldsymbol{z}_i = \begin{bmatrix} \boldsymbol{x}_i \\ \boldsymbol{y}...

**5**

votes

**2**answers

361 views

### Symmetric linear least-squares solution

Given tall matrices $A$ and $Y$ and the following overdetermined linear system in square matrix $X$
$$AX=Y$$
is there an explicit formula for the least-squares solution if $X$ is constrained to be ...

**0**

votes

**1**answer

313 views

### How to determine the damping factor in Levenberg-Marquardt?

From the algorithm, we can see that it tries different damping factor until it gets a good one by the error. Is the damping factor related to the eigenvalues of the Hessian matrix?

**2**

votes

**0**answers

19 views

### Difference Between Total Least Squares Plane and Plane Orthogonal to Principal Axis of Inertia Tensor

Given a finite set $P$ of points in $\mathbb{E}^3$ , one can calculate an approximating plane either as the solution of a Total Least Squares problem or by interpreting the problem physically, ...

**1**

vote

**1**answer

85 views

### Minimization Proof of Conditioning on Gaussian is Gaussian

It is well known that $E[X|X+Y]$ is Gaussian if both $X$ and $Y$ are, and the result can be derived using standard density arguments. However, how can one prove it by only resulting to optimization ...

**1**

vote

**0**answers

98 views

### Choice of residual function for least squares error minimization

Good morning,
I have the a set of data $(\sigma,D,\alpha_0)_i$, $i=1...n$ data.
I want to determine two parameters $K_{IC}$, $C_f$ in the basic equation given as
$K_{IC} = \sigma \sqrt{D} k_0(\...

**2**

votes

**2**answers

148 views

### In practice, what's the fastest method to find a least square solution rather than using SVD decompostion?

I'm working on a real-time implementation of Lucas-Kanade for optical flow. However, the SVD decomposition to do achieve the least square method to reduce the error seems to take too much time.
A ...

**8**

votes

**3**answers

262 views

### Regularized linear vs. RKHS-regression

I'm studying the difference between regularization in RKHS regression and linear regression, but I have a hard time grasping the crucial difference between the two.
Given input-output pairs $(x_i,y_i)...

**5**

votes

**1**answer

192 views

### Rank-constrained least-squares solution of the Sylvester matrix equation

For the Sylvester matrix equation $AX+XB=C$, I want to find the least-squares solution $X$ via
$$\begin{array}{ll} \text{minimize} & \| AX + XB - C \|_{\text{F}}^2\\ \text{subject to} & \mbox{...

**4**

votes

**1**answer

266 views

### Least square solution to $AXB+CXD=E$

I am trying to find the least-squares solution $X$ of the following matrix equation
$$AXB+CXD=E$$
Of course, I know that this equation can be written in the form
$$(B^T \otimes A+D^T \otimes C) \...

**3**

votes

**2**answers

310 views

### Standard solution to semidefinite program [closed]

I have an optimization problem of the following form
$$\text{minimize} \,\|Qa-b\|_2 \quad \text{ subject to } Q \succeq 0$$
where $a,b \in \mathbb{R}^n$ are given and the $n \times n$ square matrix ...

**2**

votes

**1**answer

171 views

### Reconstruct matrix given all differences of neighbors

We have an unknown $m\times n$ matrix $X=(x_{ij})_{i=1,j=1}^{m,n}$. Assume we are given measurements of the differences
$$x_{i,j+1}-x_{i,j}$$
and
$$x_{i+1,j}-x_{i,j}$$ for all $(i,j)\in \{1,\...

**6**

votes

**1**answer

13k views

### Gauss-Newton vs Gradient Descent vs Levenberg-Marquadt for least squared method

I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I now about this topic comes from the book of Taylor "Introduction to error analysis": a set of ...

**0**

votes

**0**answers

68 views

### Why is ideal wavelet selection a least-squares estimate?

In their classic paper "Ideal spatial adaptation by wavelet shrinkage" (http://biomet.oxfordjournals.org/content/81/3/425.short?rss=1&ssource=mfr), Donoho and Johnstone make the following ...

**1**

vote

**1**answer

103 views

### MSE of measurable function is still conditional expectation

Motivation
Then the usual stochastic filtering problem says that:
$$
\operatorname{argmin}_{Z \in L^2(\mathscr{G}_t)}\,\mathbb{E}[(Y_t-Z_t)^2],
$$
where $\mathscr{G}_t$ is the $\sigma$-algebra ...

**1**

vote

**1**answer

172 views

### Least squares problem with constrained solution [closed]

If $a_{m\times 1}$ and $Q_{m\times n}$ ($m<n $) are known, and we know every element of $b$ is between $[-1\ \ 1]$, how to determine $b$ to minimize $\|a+Qb\|_2$?

**2**

votes

**1**answer

228 views

### How to force least squares solution matrix to be diagonal? [closed]

I have the following matrix equation
$$AX=B$$
given $8 \times 3$ matrices $A$ and $B$. $X$ is a $3 \times 3$ diagonal matrix whose main diagonal contains the $3$ unknowns.
Whenever I solve for $X$ ...

**7**

votes

**1**answer

167 views

### Least-squares solution of systems of Sylvester equations

The Sylvester equation $AX+XB=C$ has been studied quite a lot and there are known algorithms for solving it.
But has the situation where (an over-determined) system of equations $A_{i}X+XB_{i}=C_{i}$ ...