Let $(X,\tau)$ be a topological space. Let us call $x,y\in X$ swappable if there is $f:X\to X$ continuous such that $f(x)=y$ and $f(y)=x$. This relation is obviously reflexive and symmetric, but not necessarily transitive.
Moreover, we call $(X,\tau)$ rigid if the identity is the only homeomorphism from $X$ to itself.
Is there a rigid space $(X,\tau)$ with $|X| > \aleph_0$ such that for all $x,y\in X$ we have that $x,y $ are swappable?