A necessary and sufficient condition for continuity is that you have two end points $a < b$ such that $f(f(a)) = f(a)$ and similarly for $b$, in other words both $f(a)$ and $f(b)$ should be fixed points of $f$. So, take any continuous function on some interval such that the graph of this function cuts the line $y=x$ at $x_0$ on that interval, now extend it continuously to any interval $[a,b]$ containing the original one such that $f(a) = f(b) = x_0$. If you have two intersection points $x_0$ and $x_1$, then you can specify the function values at $a$ and $b$ to be different. This gives you an enormous supply of examples.
As for smoothness, I believe that in fact a necessary and sufficient condition is that $f$ is smooth on $(a,b)$, that $f'(a) = \pm\infty$ and that $f'(x_0)<0$, and similarly for $b$ (and possibly $x_1$, if that's how you constructed $f$). Indeed, $\frac{d}{dt}f(f(t))|_a = f'(f(a))f'(a)=f'(x_0)f'(a)$ which will also be $\pm\infty$ but with the opposite sign, if the above condition is satisfied, and the condition is clearly necessary. For the derivative of each higher iterate, you will have a corresponding number of $f'(x_0)$ in the product, so the sign will keep swapping. So yes, your curve seems to be smooth at the end points, thanks to arccos.