Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [least-squares]

The tag has no usage guidance.

0
votes
0answers
46 views

Techniques for minimizing function [closed]

How does one obtain $W^*$ from Eq. (7) in Shichao Zhang et al, Efficient kNN Classification With Different Numbers of Nearest Neighbors, IEEE Transactions on Neural Networks and Learning Systems ...
1
vote
0answers
33 views

Choice of residual function for least squares error minimization

Good morning, I have the a set of data $(\sigma,D,\alpha_0)_i$, $i=1...n$ data. I want to determine two parameters $K_{IC}$, $C_f$ in the basic equation given as $K_{IC} = \sigma \sqrt{D} k_0(\...
2
votes
2answers
126 views

In practice, what's the fastest method to find a least square solution rather than using SVD decompostion?

I'm working on a real-time implementation of Lucas-Kanade for optical flow. However, the SVD decomposition to do achieve the least square method to reduce the error seems to take too much time. A ...
8
votes
3answers
202 views

Regularized linear vs. RKHS-regression

I'm studying the difference between regularization in RKHS regression and linear regression, but I have a hard time grasping the crucial difference between the two. Given input-output pairs $(x_i,y_i)...
5
votes
1answer
146 views

Rank-constrained least-squares solution of the Sylvester matrix equation

For the Sylvester matrix equation $AX+XB=C$, I want to find the least-squares solution $X$ via $$\begin{array}{ll} \text{minimize} & \| AX + XB - C \|_{\text{F}}^2\\ \text{subject to} & \mbox{...
4
votes
1answer
163 views

Least square solution to $AXB+CXD=E$

I am trying to find the least-squares solution $X$ of the following matrix equation $$AXB+CXD=E$$ Of course, I know that this equation can be written in the form $$(B^T \otimes A+D^T \otimes C) \...
2
votes
2answers
240 views

Standard solution to semidefinite program [closed]

I have an optimization problem of the following form $$\text{minimize} \,\|Qa-b\|_2 \quad \text{ subject to } Q \succeq 0$$ where $a,b \in \mathbb{R}^n$ are given and the $n \times n$ square matrix ...
3
votes
1answer
159 views

Reconstruct matrix given all differences of neighbors

We have an unknown $m\times n$ matrix $X=(x_{ij})_{i=1,j=1}^{m,n}$. Assume we are given measurements of the differences $$x_{i,j+1}-x_{i,j}$$ and $$x_{i+1,j}-x_{i,j}$$ for all $(i,j)\in \{1,\...
3
votes
1answer
6k views

Gauss-Newton vs Gradient Descent vs Levenberg-Marquadt for least squared method

I need to clarify some idea I have in my mind about linear and non-linear regressions. Whatever I now about this topic comes from the book of Taylor "Introduction to error analysis": a set of ...
0
votes
0answers
48 views

Why is ideal wavelet selection a least-squares estimate?

In their classic paper "Ideal spatial adaptation by wavelet shrinkage" (http://biomet.oxfordjournals.org/content/81/3/425.short?rss=1&ssource=mfr), Donoho and Johnstone make the following ...
1
vote
1answer
91 views

MSE of measurable function is still conditional expectation

Motivation Then the usual stochastic filtering problem says that: $$ \operatorname{argmin}_{Z \in L^2(\mathscr{G}_t)}\,\mathbb{E}[(Y_t-Z_t)^2], $$ where $\mathscr{G}_t$ is the $\sigma$-algebra ...
1
vote
1answer
124 views

Least squares problem with constrained solution [closed]

If $a_{m\times 1}$ and $Q_{m\times n}$ ($m<n $) are known, and we know every element of $b$ is between $[-1\ \ 1]$, how to determine $b$ to minimize $\|a+Qb\|_2$?
2
votes
1answer
148 views

How to force least squares solution matrix to be diagonal? [closed]

I have the following matrix equation $$AX=B$$ given $8 \times 3$ matrices $A$ and $B$. $X$ is a $3 \times 3$ diagonal matrix whose main diagonal contains the $3$ unknowns. Whenever I solve for $X$ ...
7
votes
1answer
132 views

Least-squares solution of systems of Sylvester equations

The Sylvester equation $AX+XB=C$ has been studied quite a lot and there are known algorithms for solving it. But has the situation where (an over-determined) system of equations $A_{i}X+XB_{i}=C_{i}$ ...