$\newcommand\tr{\operatorname{tr}}$Let $$Q(B):=(Y-XB)^T(Y-XB).$$ Since the column spaces of the matrices $X^TX$ and $X^T$ are the same, there is a matrix $B_*$ such that $$X^TXB_*=X^TB_*.$$ For each $z\in\Bbb R^{p\times}$, by (say) differentiating the convex function $B\mapsto z^T Q(B)z$ in $B$, we see that $B_*$ is a minimizer of this function. So, $B_*$ is a minimizer of $Q(B)$ in $B$ wrt to the Loewner ordering. So, $B_*$ is a minimizer of both $\tr Q(B)$ and $\det Q(B)$ in $B$.
Generically, $\tr Q(B)$ is strictly convex in $B$, and then $B_*$ is a unique minimizer of $\tr Q(B)$. However, a minimizer of $\det Q(B)$ is not unique.