Summarizing a discussion in the comments on Harry's answer, we can consider the case $\mathcal C=\mathrm{Gpd}$, with the canonical model structure: weak equivalences are just the equivalences of groupoids, cofibrations are functors injective on objects, and fibrations are isofibrations.
A $\mathcal C$-enriched category with one object is then precisely a strictly monoidal groupoid. In contrast, a $\mathrm{Ho}(\mathcal C)$-enriched category with one object is essentially the same thing as an $A_3$-monoidal groupoid-we get a tensor product, which is certainly associative up to non-specified isomorphism, but there is no reason why the associators should satisfy the pentagon identity, and indeed they need not.
For a counterexample, consider the $A_3$-monoidal groupoid $G$ freely generated by a single object $x$. For simplicity, let's also make it strictly unital. Then the objects of $G$ are given by rooted binary trees, corresponding to parenthesizations of finite strings of $x$'es. The morphisms are freely generated by the associator isomorphism between the two rooted binary trees of three leaves, together with the functoriality of the tensor product $\otimes:G\times G\to G$ which fuses two trees by adding a new root to their disjoint union.
It is in fact possible to describe the morphisms in $G$ without reference to the tensor product: they are sequences of elementary moves, where an elementary move out of a binary tree $T$ is given by taking a node $N$ which is the left child of its parent $P$ and replacing $P$'s right child with $N$, $P$'s left child with $N$'s left child, $N$'s left child with its right child, and $N$'s right child with $P$'s right child. Under this description, one can see that the two different paths from $((xx)x)x$ to $x(x(xx))$ around the pentagon are not equal in $G$. (For comparison, the free strictly unital $A_4$-monoidal groupoid on $x$ has exactly one morphism between two binary trees whenever there exists any such sequence of elementary moves transforming one into the other.)
Thus $G$ gives an $A_3$-monoidal groupoid which does not arise from an $A_4$-monoidal one, equivalently, not from a strictly monoidal one. EDIT: As Harry notes, this is not quite a counterexample to your question, as it's not clear that it's impossible to make a different choice of the associators that would extend to $A_4$. It seems unlikely, but also that it would be complicated to make an elementary argument against it.