The nice answer of Alexey Ustinov shows that the OP's second sum equals, up to a $p$-th root of unity, a Kloosterman sum $K(a,b;p)$, where $a$ mod $p$ and $b$ mod $p$ depend only on $l$ mod $p$. However, we also know that $K(a,b;p)\neq 0$ by Noam Elkies's response here (see also the comments of Lucia and KConrad there), so the OP's second sum is nonzero as well.
It follows that the OP's second sum is at least as large (in absolute value) as the shortest nonzero element of the ring $\mathbb{Z}[\omega_p]$. The latter quantity has been discussed at many places (e.g. at MO here), so hopefully someone can add a good reference and a reasonable lower bound.