# Questions tagged [least-squares]

The tag has no usage guidance.

26 questions
Filter by
Sorted by
Tagged with
0answers
23 views

### Nonlinear least-squares parameter estimation of a vector-valued function

I'm given a function $f\, (x; \vec{\beta})$, non-linear, where ${x} \in \mathbb{R}$ and $\vec{\beta} \in \mathbb{R}^k$. It models an experiment of a setup characterized by the parameters $\vec{\beta}$....
4answers
999 views

### Is the pseudoinverse the same as least squares with regularization?

Given a linear system $Ax=b$, the pseudoinverse of $A$ is found as the matrix $A^+$ such that $x=A^+ b$ where $x$ solves the least squares problem $\min \| Ax - b \|^2$ and $x \perp \mathcal{N}(A)$. ...
1answer
142 views

### Concentration inequality for norm of solution to nonlinear least-squares problem

Define the piecewise-linear function $\psi(t):=\max(t,0)$ for all $t \in \mathbb R$. Let $d,n,k \to \infty$ at the same rate (i.e $n \asymp k \asymp d$). Let $y_1,\ldots,y_n \in \{-1,1\}$ uniformly ...
2answers
114 views

### Robust estimation of $Ax=b$

Problem setting : $\underset{x}{\text{min}} \|Ax-b\|$, where $A \in \mathcal{R}^{m \times n}, m\gg n$, full rank. L1 loss is used for robust estimation using IRLS. The corresponding equation to ...
1answer
76 views

### Understanding the Time Delay of Arrival trilateration algorithm

I'm trying to algorithmically solve the Time Delay of Arrival problem as part of some mathematics research. The problem is as follows: Given the location of three receivers in a plane (A, B, and C), ...
0answers
41 views

### unusual least square solution in SINDy algorithm

In the SINDy algorithm, the equation $\dot{X} \approx \Theta\Xi$ has to be solved for $\Xi$. $\dot{X}$ and $\Theta$ are known numerical matricies. The intuitive solution to this problem would be to ...
2answers
63 views

### Non-parametric regression and curvature

Given a finite set of points $(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)$ in the plane, Linear Regression tells us how to find the straight line "$y=a+bx$" best approximating the given points, in the ...
3answers
70 views

### RKHS/non-parametric regression with missing response values

I am interested in doing RKHS regression with missing response variables. Given input-output pairs $(x_i,y_i)$, I want to estimate a function $f(\cdot)$ as follows \begin{equation}f(x)\approx u(x)=\...
1answer
288 views

2answers
148 views

### In practice, what's the fastest method to find a least square solution rather than using SVD decompostion?

I'm working on a real-time implementation of Lucas-Kanade for optical flow. However, the SVD decomposition to do achieve the least square method to reduce the error seems to take too much time. A ...
3answers
262 views

1answer
13k views

### Gauss-Newton vs Gradient Descent vs Levenberg-Marquadt for least squared method

I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I now about this topic comes from the book of Taylor "Introduction to error analysis": a set of ...
0answers
68 views

### Why is ideal wavelet selection a least-squares estimate?

In their classic paper "Ideal spatial adaptation by wavelet shrinkage" (http://biomet.oxfordjournals.org/content/81/3/425.short?rss=1&ssource=mfr), Donoho and Johnstone make the following ...
1answer
103 views

### MSE of measurable function is still conditional expectation

Motivation Then the usual stochastic filtering problem says that: $$\operatorname{argmin}_{Z \in L^2(\mathscr{G}_t)}\,\mathbb{E}[(Y_t-Z_t)^2],$$ where $\mathscr{G}_t$ is the $\sigma$-algebra ...
1answer
172 views

### Least squares problem with constrained solution [closed]

If $a_{m\times 1}$ and $Q_{m\times n}$ ($m<n$) are known, and we know every element of $b$ is between $[-1\ \ 1]$, how to determine $b$ to minimize $\|a+Qb\|_2$?
1answer
228 views

### How to force least squares solution matrix to be diagonal? [closed]

I have the following matrix equation $$AX=B$$ given $8 \times 3$ matrices $A$ and $B$. $X$ is a $3 \times 3$ diagonal matrix whose main diagonal contains the $3$ unknowns. Whenever I solve for $X$ ...
1answer
167 views

### Least-squares solution of systems of Sylvester equations

The Sylvester equation $AX+XB=C$ has been studied quite a lot and there are known algorithms for solving it. But has the situation where (an over-determined) system of equations $A_{i}X+XB_{i}=C_{i}$ ...