$\newcommand\si\sigma\newcommand\bm[1]{\begin{bmatrix}#1\end{bmatrix}}$No and no: In general, (i) $EB\ne0$ and (ii) we cannot bound $Var\,\|B\|_F^2$ by $\si^2$.
E.g., let $n=3$, $d=1$, $$y_1:=\bm{1\\ -1\\ -1},\quad y_2:=\bm{-1\\ 1\\ -1},\quad y_3:=\bm{-1\\ -1\\ 1},\quad y_4:=\bm{1\\ 1\\ 1}, $$ $$A:=\bm{1&-1&0\\0&1&0\\0&0&1}.$$ Let $Y$ be a random matrix such that $P(Y=y_j)=1/4$ for $j=1,2,3,4$. Let $$X:=A^{-1}Y.$$
Then $EAX=EY=0$ and $\|AX\|_F^2=\|Y\|_F^2=3$ almost surely (a.s.), so that $$\si^2=Var\,\|AX\|_F^2=0.$$
However, $$EB=\bm{0\\ 0\\ -1/6}\ne0.$$ Also, the values of $\|B\|_F^2$ at $Y=y_1$ and at $Y=y_4$ are the non-equal numbers $2$ and $8/3$, respectively, so that $Var\,\|B\|_F^2$ is strictly greater than $0=\si^2$. (In fact, $Var\,\|B\|_F^2=1/9$.)