Thank you for good examples! Probably it should be made more precise what kind of $X$ and $ f$ are relevant. I've also considered a similar cubic polynomial, but with the domain of definition being the whole $ X =\mathbb {R} $ instead of an interval. In that case, $ E (f)$ is connected (being the union of an oval and a diagonal line passing through the oval). The example you construct can be thought as a cutoff of the set $E(f)$ above. Thus, some 'completteness' of X is needed. Another relevant example is $ f=\sin x\sin y:\,\mathbb {R}^2\to\mathbb {R}$ for which $ E(f) $ is connected again.

P.S. Notice that the target space is $\mathbb {R} $, it might be crucial.

The examples I have in mind, all of them have the property that "there exists at least one $ a\in \mathbb { R } $ such that $ f^{-1}(a) $ is connected".

A nice example! What is about $ f:\mathbb{R}^n\to\mathbb{R}$. My original problem comes from a rather concrete $ f:\mathbb{R}^4\to\mathbb{R} $ for which the statement is true, even in a considerably larger class of functions.