Update: A bit of a digital paper chase led me, via David Robert's thesis (note that in the latest version, it is Chapter 5, section 2 that is most relevant), to this paper on the arXiv. The last sentence of the abstract is:
These hoop earring spaces provide a simple class of counterexamples to the claim that $\pi_{1}^{top}$ is a functor to the category of topological groups.
I recommend reading this article.
(Added later: In case it's not clear, the author of that paper is Jeremy Brazas who added an answer afterwards, so if you vote for my answer, you should definitely vote for his!)
The proof given in the second paper is short enough that I think it reasonable to copy it out here. I shan't copy out the obvious diagram so need to establish some notation first:
- $m \colon \pi_1^{Top}(X,x) \times \pi_1^{Top}(X,x) \to \pi_1^{Top}(X,x)$ is the multiplication map in question
- $p \colon \operatorname{Hom}((S^1,1),(X,x)) \to \pi_1^{Top}(X,x)$ is the quotient map
- $\overline{m} \colon \operatorname{Hom}((S^1,1),(X,x)) \times \operatorname{Hom}((S^1,1),(X,x)) \to \operatorname{Hom}((S^1,1),(X,x))$ is the "upstairs" multiplication map. (It's a tilde in the original, but that isn't displaying correctly for me so I daren't use it.)
The proof then proceeds:
To show that $m$ is continuous, it suffices to show that $\overline{m}$ is continuous, for then if $U \subset \pi_1^{Top}(X,x)$ is open, $(p \times p)^{-1} m^{-1}(U) = \overline{m}^{-1}p^{-1}(U)$ is open, but by the definition of a quotient map, $(p \times p)^{-1} m^{-1}(U)$ is open if and only if $m^{-1}(U)$ is.
There then follows a proof that $\overline{m}$ is continuous, a fact that I trust does not need proving.
Comments on your comments:
We don't need the quotient map to be open since we are only ever dealing with preimage sets. It is certainly not always true that if $q \colon X \to Y$ is a quotient that $q(U)$ is open in $Y$ for every open $U$ in $X$. But it is true by definition that $q^{-1}(U)$ is open in $X$ if and only if $U$ is open in $Y$. This is because the topology on $Y$ is precisely that to make this true. So since we are only dealing with sets of the form $(p \times p)^{-1}(A)$ then the assertion is valid assuming that $p \times p$ is a quotient map.
Here, I find myself worried. A quick back-of-envelope check seems to show that one can't simply assume that the product of quotients is again a quotient in Top (a counterexample eludes me as I don't have Counterexamples in Topology to hand and I'm too used to dealing with "nice" spaces). It may be the case that for Hom-spaces then there's some magic that can be done (though such is not mentioned in the paper); but again the best that I can do on the back of an envelope is observe that (modulo some basepoint mess) by construction $\operatorname{Hom}((S^1,1),(X,x)) \times \operatorname{Hom}((S^1,1),(X,x))$ quotients to $\pi_1^{Top}((X,x) \times (X,x))$. But to proceed, one would need to know that $\pi_1^{Top}$ was a product-preserving functor. This is morally the same as saying that it is representable - which looks good since we have an obvious representing object $S^1$! However, this can't be made into a proper argument since although we have a representing object, we don't have an enriched Hom-functor $hTop \times hTop \to Top$ which to evaluate at $S^1$.
So I would look for a counterexample to the product of quotients being a quotient, and see where that leads you. Either you'll find a proper counterexample to the proposition in question, or you'll see why in this special case, such a counterexample could not occur.
(Of course, I may well be missing something obvious!)