2
$\begingroup$

Consider the multivariate regression model $$Y = XB + E$$ where $Y$ is $n \times p$ and corresponds to the dependent variables, $X$ is $n \times k$ and corresponds to the independent variables, $B$ is $k \times p$ and corresponds to the coefficients that we wish to estimate. Lastly, let $E \sim N_{n, p}(0, \Sigma, I_n)$ correspond to the errors in the model. We can directly note that $(Y-XB)^{T}(Y-XB) \succ 0$.

I wonder if the following equivalence holds: \begin{align} & \operatorname*{argmin}_B \operatorname{trace}(Y-XB)^{T}(Y-XB) \\ = {} & \operatorname*{argmin}_B \det(Y-XB)^{T}(Y-XB). \end{align}

I have done some simulations using real data which seems to indicate that it is positive, but I'm very skeptical (since the left hand side is convex in $B$ and the right hand side isn't). Some aspects I have already considered are:

  • Since we have strong convexity of the left hand side, we know that a minimizer exists. Denoting this by $B^{*}$ we have the sufficient condition $$(Y-XB^{*})^T(Y-XB^{*}) \preceq (Y-XB)^T(Y-XB)$$ which is also quite hard to prove (in the case that it is true).

  • Using that $\det(e^{(Y-XB)^{T}(Y-XB)}) = e^{\operatorname{trace}(Y-XB)^T(Y-XB)}$ and considering the corresponding eigenvalues.

I would love any insights or counter-examples to the equivalence statement.

$\endgroup$
2
  • 1
    $\begingroup$ When you write \text{det} A rather than \det A then you see $\text{det} A$ rather than $\det A.$ This doesn't just mean that \det provides horizontal spacing; rather the spacing depends on the context, so that \det (A) gives you $\det(A),$ with a smaller amount of space to the right. And this also applies to the left side. The same applies to using \text{trace} rather than \operatorname{trace}. And in \operatorname*{argmin}, the asterisk affects the formatting of the subscript when in a "displayed" (as opposed to "inline") context. $\endgroup$ Commented Apr 22 at 17:51
  • $\begingroup$ $$ \begin{align} & \text{det} A \\ \text{versus } \qquad & {\det A} \end{align}$$ $\endgroup$ Commented Apr 22 at 17:52

1 Answer 1

1
$\begingroup$

$\newcommand\tr{\operatorname{tr}}$Let $$Q(B):=(Y-XB)^T(Y-XB).$$ Since the column spaces of the matrices $X^TX$ and $X^T$ are the same, there is a matrix $B_*$ such that $$X^TXB_*=X^T Y.$$ For each $z\in\Bbb R^{p\times1}$, by (say) differentiating the convex function $B\mapsto z^T Q(B)z$ in $B$, we see that $B_*$ is a minimizer of this function. So, $B_*$ is a minimizer of $Q(B)$ in $B$ wrt to the Loewner ordering. So, $B_*$ is a minimizer of both $\tr Q(B)$ and $\det Q(B)$ in $B$.

Generically, $\tr Q(B)$ is strictly convex in $B$, and then $B_*$ is a unique minimizer of $\tr Q(B)$. However, a minimizer of $\det Q(B)$ is not unique.

For instance, suppose that $$X=\left( \begin{array}{cc} 1 & 3 \\ 1 & 1 \\ \end{array} \right), \quad Y=\left( \begin{array}{cc} 3 & 0 \\ 2 & 1 \\ \end{array} \right).$$ Then $$B_*=\frac12\,\left( \begin{array}{cc} 3 & 3 \\ 1 & -1 \\ \end{array} \right)$$ is the unique minimizer of $\tr Q(B)$. However, $$\left( \begin{array}{cc} 0 & 0 \\ 0 & -1 \\ \end{array} \right)$$ is a minimizer of $\det Q(B)$.

$\endgroup$
3
  • $\begingroup$ Thank you. Since the Loewner order is only a partial ordering, can we actually make the conclusion that $B_{*}$ is a minimizer (wrt. the ordering) over all $k \times p$ matrices $B$? $\endgroup$ Commented Apr 23 at 7:13
  • $\begingroup$ @respectableuser1 : It is shown here that $Q(B_*)\lesssimeq Q(B)$ for all $B$, where $\lesssimeq$ is the Loewner order. That $\lesssimeq$ is a partial order does not diminish this fact. Somewhat similarly, $0\lesssimeq B^T B$ for all $B$, even though $\lesssimeq$ is only a partial order. Do you see it now? $\endgroup$ Commented Apr 24 at 1:24
  • $\begingroup$ Yes I see it. Thank you very much for your time. $\endgroup$ Commented Apr 24 at 7:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.