Suppose $BS(1,n)$ is the BaumslagSolitar group and $S_m$ is the symmetric group. If $\Phi: BS(1,n) \to S_m$ is a homomorphism, must the image of $\Phi$ be abelian?

3$\begingroup$ It's possibly reasonable to move the question to MathSE, but clearly absurd to vote to close as "off topic because this question does not appear to be about research level mathematics within the scope defined in the help center", as someone did. $\endgroup$– YCorJan 3, 2018 at 2:43

2$\begingroup$ BS(1,n) is linear, hence residually finite (by Malcev), hence has nonabelian quotients. A faithful linear representation is given on the Wikipedia page. en.wikipedia.org/wiki/… $\endgroup$– Ian AgolJan 3, 2018 at 18:59

1$\begingroup$ I would also note that the fact that ${\rm BS}(1,2)$ maps onto $S_3$ is clear. $\endgroup$– Derek HoltJan 3, 2018 at 22:35
3 Answers
No, these groups have many finite nonabelian quotients. Recall that the BaumslagSolitar group $B(1,n)$ has a presentation $$ B(1,n)=\langle t,a\,\,tat^{1}=a^n\rangle. $$ A homomorphism $\Phi: B(1,n)\to S_m$ amounts to a pair $(\tau,\alpha)$ of permutations, where $\Phi(a)=\alpha$ and $\Phi(t)=\tau$, such that $\tau$ conjugates $\alpha$ to $\alpha^n$.Observe that in the symmetric group $S_m$, a cycle $\alpha$ of length $k$ is conjugate to its $n$th power $\alpha^n$ provided $n$ and $k$ are relatively prime, and $\alpha^n\ne \alpha$ for $k$ not dividing $n1$, so that the conjugating permutation $\tau$ does not commute with $\alpha$. This yields examples of homomorphisms with nonabelian image.
For a specific example, take $n=2, k=3$ and let $\Phi: B(1,2)\to S_3$ map the
generators $t$ and $a$ of
$$
B(1,2)=\langle t,a\,\,tat^{1}=a^2\rangle
$$
into permutations $\tau=\Phi(t)=(2,3)$ and $\alpha=\Phi(a)=(1,2,3)$ in the cycle notation. Since $\Phi(t)=(2,3)$ conjugates $\Phi(a)=(1,2,3)$ into its square $\Phi(a)^2=(1,3,2)$ and moreover, $(2,3)$ and $(1,2,3)$ generate $S_3$, this defines a group homomorphism whose image is $S_3$.
The answer is no. Since every finite group embeds in $S_m$ for some $m$, this is equivalent to asking if every finite quotient of $BS(1,n)$ is abelian. The finite quotients of $BS(1,n)$ are all the quotients of groups of the form $\mathbb{Z}/s\mathbb{Z} \rtimes \mathbb{Z}/r\mathbb{Z}$ where the action is multiplication by $n$ and $n^r \equiv 1 \bmod s$. See this paper by Matei and Suciu for example.
Another way to see the answer is no: the dihedral group $D_{2k}$ (sometimes written $D_k$) of order $2k$ is generated by $a$ ('rotation through $2\pi/k$') and $t$ (a reflection) and satisfies the relation $tat^{1}=a^{1}=a^{k1}$. This is clearly a finite nonabelian group provided $k>2$, and taking $k=n+1$, we see that $D_{2k}$ is an image of $\mathrm{BS}(1,n)$ under the obvious map.
Finally, since $D_{2k}$ acts on the vertices of a regular $k$gon, $D_{2k}$ embeds in $S_m$ for any $m\geq k$. This gives a negative answer to the question provided $n\geq 2$ and $m\geq 3$.