Let $\mathcal{C}$ be a minimal cyclic $A_{\infty}$ category, as considered in, say Kajiura's thesis, or Kontsevich and Soibelman's deformation theory notes, or http://arxiv.org/abs/math/0412149. I.e. it's a minimal $A_{\infty}$ category in the usual sense, with a nondegenerate symmetric pairing $\langle\bullet,\bullet\rangle$ between $\mathrm{Ext}^i(x,y)$ and $\mathrm{Ext}^{n-i}(y,x)$ such that $\langle m_s(\bullet,\ldots,\bullet),\bullet\rangle$ is cyclically invariant (up to the usual sign horror) for every $s$, where the $m_s$ are the (higher) composition operations in the category $\mathcal{C}$. The number $n$ here is fixed throughout.
Now say that $x$ and $y$ are isomorphic objects of $\mathcal{C}$, in the sense that there are morphisms $f:x\rightarrow y$ and $g:y\rightarrow x$, such that $m_2(f,g)=\mathrm{id}_y$ and $m_2(g,f)=\mathrm{id}_x$.
My question is: is there a quick proof that $\mathrm{End}_{\mathcal{C}}(x)$ and $\mathrm{End}_{\mathcal{C}}(y)$ are isomorphic as cyclic $A_{\infty}$ algebras? E.g. we want a morphism $F$ of $A_{\infty}$ algebras such that $F_1$ pulls back the pairing $\langle\bullet,\bullet\rangle$ and there are equalities $\sum_{i+j=t}\langle F_i,F_j\rangle=0$ for all $t\geq 3$. While this is hopefully true, I can't find a reference anywhere, and unless f and g are strict, in the sense that they vanish for all higher compositions, I don't see a quick proof.