Let $p(x) = \sum_{n=1}^N e^{2 \pi i a_n x}$ be a trigonometric polynomial, where $a_n$ are distinct positive integers. There is a classical trick which (using Hölder's inequality) allows to give a lower bound for the $L^1$ norm of $p$ in terms of the $L^4$ norm of $p$. One obtains $$ \|p\|_1 \geq \frac{\|p\|_2^{3}}{\|p\|_4^{2}} = \frac{N^{3/2}}{\|p\|_4^{2}}. $$ So roughly speaking a small $L^4$ norm implies a large $L^1$ norm.
Question: Is the opposite also true? That is, does a large $L^4$-norm imply a small $L^1$-norm? (And if "yes", is there a quantitative estimate?)
(This might be a stupid question, but still I am grateful for an answer.)
(PS: For a reference to the trick mentioned above, see for example A. A. Karatsuba, "An estimate of the L1-norm of an exponential sum", Mathematical Notes, 1998, 64:3, 401-404.)