Let $X,Y$ be real separable Hilbert space, and let $HS(X,Y)$ be the space of Hilbert-Schmidt operators from $X$ to $Y$, endowed with the Hilbert-Schmidt norm. Let $x\in X$ and $y\in Y$. I am interested in finding the derivative of the map $$ HS(X,Y)\ni A\mapsto \|y-Ax\|_Y^2\in \mathbb{R}. $$
As $HS(X,Y)$ is a Hilbert space, the derivative (as a bounded linear operator from $HS(X,Y)$ to $\mathbb{R}$) can be identified as an element in $HS(X,Y)$. Is there an easy expression of it?
Let me denote by $F : HS(X,Y) \to \mathbb{R}$ the map which you defined as $F(A) := \|y-Ax\|_Y^2$. By standard arguments, you indeed have that $F$ is $C^1$ and, for any $A \in HS(X,Y)$, $$ \mathcal{L}(HS(X,Y),\mathbb{R}) \ni DF_{\rvert A} = \begin{cases} HS(X,Y) \to \mathbb{R}, \\ B \mapsto 2 \langle Bx , Ax - y \rangle_Y. \end{cases} $$ As you recall, $HS(X,Y)$ is a Hilbert space for the scalar product $$ \langle A, B \rangle_{HS} := \sum_{i \in I} \langle A e_i, B e_i \rangle_Y $$ for any orthonormal basis $(e_i)_{i \in I}$ of $X$.
Given $A \in HS(X,Y)$, by the Riesz representation theorem, we look for some $C \in HS(X,Y)$ such that $DF_{\rvert A} = \langle C, \cdot \rangle_{HS}$. Thus, we must have, for all $B \in HS(X,Y)$, $$ \begin{split} \sum_{i \in I} \langle C e_i, B e_i \rangle_Y & = 2 \langle Bx, Ax-y \rangle_Y \\ & = 2 \left\langle B \left(\sum_{i\in I} \langle x, e_i \rangle_X e_i \right), Ax-y \right\rangle_Y \\ & = \sum_{i\in I} \left \langle Be_i, 2 \langle x, e_i \rangle_X (Ax-y) \right \rangle_Y. \end{split} $$ Hence, one can define $C$ by $C e_i := 2 \langle x, e_i \rangle_X (Ax-y)$. Then $C \in HS(X,Y)$ and $\|C\|_{HS} = 2 \|x\|_X \| Ax-y\|_Y$. Eventually, one can thus identify the derivative as $$ HS(X,Y) \ni DF_{\rvert A} : \begin{cases} X \to Y, \\ z \mapsto 2 \langle x, z \rangle_X (Ax-y). \end{cases} $$
