Let $K_1$ and $K_2$ be two knots such that for all finite quandles $X$, the number of colorings of $K_1$ by $X$ is the same as the number of colorings of $K_2$ by $X$. Then my question is, must $K_1$ and $K_2$ either be the same knot or mirror images of each other?

If not, does anyone know of a counterexample?