Return to Answer

Note that $A$ only appears in the combination $M=A-BC$, so the derivative with respect to $A$ equals the derivative with respect to $M$; The function $f(A)$ is given by $$f(A)=||(A - BC)^Nv - w||_2^2=(M^Nv)^T M^Nv+w^Tw-2w^TM^Nv.$$ For a simple case, let me first consider a scalar perturbation, $f(A+\epsilon I)=f(A)+\epsilon df/dA$, with derivative $$\frac{df}{dA} =2N(M^Nv-w)^TM^{N-1}v.$$

Next consider the derivative with respect to a given matrix element $A_{ij}$ of $A$. The expressions are more lengthy, basically each matrix $M$ gives a separate term so we have a sum $\sum_{k=1}^N$ instead of the factor $N$: $$\frac{\partial f}{\partial A_{ij}}= 2\sum_{k=1}^N\sum_{p,q=1}^n (M^Nv-w)_q (M^{k-1})_{qi} (M^{N-k})_{jp} v_p.$$ Note that the scalar perturbation is the trace of the matrix of elementwise perturbations, $df/dA=\sum_{i=1}^n \partial f/\partial A_{ii}$.

As noted in the comment, my result for the elementwise derivative does not seem to agree with the one from an online calculator. Let me check a simple case, $N=2$, $w=0$, $v_p=\delta_{p1}$. Then $$f=\sum_{p,q,r}M_{pq}M_{q1}M_{pr}M_{r1}.$$ Direct evaluation of the derivative with respect to $M_{22}$ gives $$\frac{\partial f}{\partial M_{22}}=2M_{21}(M^2)_{21},$$ in agreement with the general formula above. The online calculator would give 0.

Note that $A$ only appears in the combination $M=A-BC$, so the derivative with respect to $A$ equals the derivative with respect to $M$; The function $f(A)$ is given by $$f(A)=||(A - BC)^Nv - w||_2^2=(M^Nv)^T M^Nv+w^Tw-2w^TM^Nv.$$ For a simple case, let me first consider a scalar perturbation, $f(A+\epsilon I)=f(A)+\epsilon df/dA$, with derivative $$\frac{df}{dA} =2N(M^Nv-w)^TM^{N-1}v.$$

Next consider the derivative with respect to a given matrix element $A_{ij}$ of $A$. The expressions are more lengthy, basically each matrix $M$ gives a separate term so we have a sum $\sum_{k=1}^N$ instead of the factor $N$: $$\frac{\partial f}{\partial A_{ij}}= 2\sum_{k=1}^N\sum_{p,q=1}^n (M^Nv-w)_q (M^{k-1})_{qi} (M^{N-k})_{jp} v_p.$$ Note that the scalar perturbation is the trace of the matrix of elementwise perturbations, $df/dA=\sum_{i=1}^n \partial f/\partial A_{ii}$.

Note that $A$ only appears in the combination $M=A-BC$, so the derivative with respect to $A$ equals the derivative with respect to $M$; The function $f(A)$ is given by $$f(A)=||(A - BC)^Nv - w||_2^2=(M^Nv)^T M^Nv+w^Tw-2w^TM^Nv.$$ For a simple case, let me first consider a scalar perturbation, $f(A+\epsilon I)=f(A)+\epsilon df/dA$, with derivative $$\frac{df}{dA} =2N(M^Nv-w)^TM^{N-1}v.$$

Next consider the derivative with respect to a given matrix element $A_{ij}$ of $A$. The expressions are more lengthy, basically each matrix $M$ gives a separate term so we have a sum $\sum_{k=1}^N$ instead of the factor $N$: $$\frac{\partial f}{\partial A_{ij}}= 2\sum_{k=1}^N\sum_{p,q=1}^n (M^Nv-w)_q (M^{k-1})_{qi} (M^{N-k})_{jp} v_p.$$ Note that the scalar perturbation is the trace of the matrix of elementwise perturbations, $df/dA=\sum_{i=1}^n \partial f/\partial A_{ii}$.

Note that $A$ only appears in the combination $M=A-BC$, so the derivative with respect to $A$ equals the derivative with respect to $M$; The function $f(A)$ is given by $$f(A)=||(A - BC)^Nv - w||_2^2=(M^Nv)^T M^Nv+w^Tw-2w^TM^Nv.$$ For a simple case, let me first consider a scalar perturbation, $f(A+\epsilon I)=f(A)+\epsilon df/dA$, with derivative $$\frac{df}{dA} =2N(M^Nv-w)^TM^{N-1}v.$$

Next consider the derivative with respect to a given matrix element $A_{ij}$ of $A$. The expressions are more lengthy, basically each matrix $M$ gives a separate term so we have a sum $\sum_{k=1}^N$ instead of the factor $N$: $$\frac{\partial f}{\partial A_{ij}}= 2\sum_{k=1}^N\sum_{p,q=1}^n (M^Nv-w)_q (M^{k-1})_{qi} (M^{N-k})_{jp} v_p.$$ Note that the scalar perturbation is the trace of the matrix of elementwise perturbations, $df/dA=\sum_{i=1}^n \partial f/\partial A_{ii}$.

As noted in the comment, my result for the elementwise derivative does not seem to agree with the one from an online calculator. Let me check a simple case, $N=2$, $w=0$, $v_p=\delta_{p1}$. Then $$f=\sum_{p,q,r}M_{pq}M_{q1}M_{pr}M_{r1}.$$ Direct evaluation of the derivative with respect to $M_{22}$ gives $$\frac{\partial f}{\partial M_{22}}=2M_{21}(M^2)_{21},$$ in agreement with the general formula above. The online calculator would give 0.

Note that $A$ only appears in the combination $M=A-BC$, so the derivative with respect to $A$ equals the derivative with respect to $M$; The function $f(A)$ is given by $$f(A)=||(A - BC)^Nv - w||_2^2=(M^Nv)^T M^Nv+w^Tw-2w^TM^Nv.$$ For a simple case, let me first consider a scalar perturbation, $f(A+\epsilon I)=f(A)+\epsilon df/dA$, with derivative $$\frac{df}{dA} =2N(M^Nv-w)^TM^{N-1}v.$$

Next consider the derivative with respect to a given matrix element $A_{ij}$ of $A$. The expressions are more lengthy, basically each matrix $M$ gives a separate term so we have a sum $\sum_{k=1}^N$ instead of the factor $N$. It is convenient to denote the transpose by $M^T\equiv\tilde{M}$. We arrive at $$\frac{\partial f}{\partial A_{ij}}=2\sum_{k=1}^N(\tilde{M}^{k-1}M^N v)_i (M^{N-k} v)_j-2\sum_{k=1}^N (\tilde{M}^{k-1} w)_i (M^{N-k} v)_j.$$ Note that the scalar perturbation is the trace of the matrix of elementwise perturbations, $df/dA=\sum_{i=1}^n \partial f/\partial A_{ii}$.

Note that $A$ only appears in the combination $M=A-BC$, so the derivative with respect to $A$ equals the derivative with respect to $M$; The function $f(A)$ is given by $$f(A)=||(A - BC)^Nv - w||_2^2=(M^Nv)^T M^Nv+w^Tw-2w^TM^Nv.$$ For a simple case, let me first consider a scalar perturbation, $f(A+\epsilon I)=f(A)+\epsilon df/dA$, with derivative $$\frac{df}{dA} =2N(M^Nv-w)^TM^{N-1}v.$$

Next consider the derivative with respect to a given matrix element $A_{ij}$ of $A$. The expressions are more lengthy, basically each matrix $M$ gives a separate term so we have a sum $\sum_{k=1}^N$ instead of the factor $N$: $$\frac{\partial f}{\partial A_{ij}}= 2\sum_{k=1}^N\sum_{p,q=1}^n (M^Nv-w)_q (M^{k-1})_{qi} (M^{N-k})_{jp} v_p.$$ Note that the scalar perturbation is the trace of the matrix of elementwise perturbations, $df/dA=\sum_{i=1}^n \partial f/\partial A_{ii}$.