Probably there's a much less artificial example, and even more probably there's a much simpler one, but ...
Let $A=k[\varepsilon]/(\varepsilon^2)$, where $k$ is a field of characteristic two (purely so I don't need to bother about signs).
Let $\mathcal{A}$ be the category whose objects are pairs $(M,N)$ of $A$-modules and whose maps $\alpha:(M,N)\to(M',N')$ are matrices $\begin{pmatrix}\alpha_1&0\\\alpha_2&\alpha_3\end{pmatrix}$ where $\alpha_1:M\to M'$, $\alpha_2:M\to N'$, $\alpha_3:N\to N'$ are $A$-module homomorphisms.
Let $\mathcal{A}_0$ be the (non-full) subcategory with the same objects, but with only those maps $\alpha$ where $\alpha_2=0$.
For $k\geq0$, let $X(k)$ be the chain complex $$\dots\to 0\to (A,0)\to (A,0)\to\dots\to(A,0)\to(A,0)\to0\to\dots$$ with $(A,0)$ in degrees $k+1,\dots,0$, where all the non-trivial differentials are $\begin{pmatrix}\varepsilon&0\\ 0&0\end{pmatrix}$, and let $Y(k)$ be the complex $$\dots\to 0\to (0,A)\to (0,A)\to\dots\to(0,A)\to0\to0\to\dots$$ with $(0,A)$ in degrees $k+1,\dots,1$, where all the non-trivial differentials are $\begin{pmatrix}0&0\\ 0&\varepsilon\end{pmatrix}$.
Consider the chain map (over $\mathcal{A}$) $f(k):X(k)\to Y(k)$ given by $\begin{pmatrix}0&0\\ \varepsilon&0\end{pmatrix}$ in degree $k+1$ and zero in other degrees. This is null-homotopic (for example, take the homotopy given by $\begin{pmatrix}0&0\\1&0\end{pmatrix}$ in degrees $k,\dots,0$), but there's no choice of contracting homotopy with degree zero component in $\mathcal{A}_0$.
Now let $\mathcal{A}^{\mathbb{N}}$ be the category of $\mathbb{N}$-graded objects of $\mathcal{A}$, and consider the chain map $$(f(k))_{k\in\mathbb{N}}:(X(k))_{k\in\mathbb{N}}\to (Y(k))_{k\in\mathbb{N}}.$$
As a map of chain complexes over $\mathcal{A}^{\mathbb{N}}$ this is null-homotopic.
However, if you regard it a map of chain complexes over the (non-full) subcategory of $\mathcal{A}^{\mathbb{N}}$ with the same objects but where maps are required to have grade $d$ components in $\mathcal{A}_0$ for all but finitely many $d$, then it isn't null-homotopic but (using contracting homotopies for finitely many $f(k)$) is homotopic to a map that is zero in any given finite set of degrees.