Let $k$ be a field of characteristic $p$ and $G$ a finite group. This question might be a dulicate of this question:
Which finite groups have no irreducible representations other than characters?
but I think it is still something that remained to explain.
If $G$ has all irreducible representations one dimensional then $G$ is an extension of a $p$-group (namely $G'$) by an abelian group as Ehud proved in his answer.
It still remains to show that this extension splits in order to deduce that $G$ is a semidirect product. Could you please explain why this extension should split?
I couldn't understand Pablo's the argument with the triangularizable matrices.