Let $k$ be a field of characteristic $p$ and $G$ a finite group. This question might be a dulicate of this question:

Which finite groups have no irreducible representations other than characters?

but I think it is still something that remained to explain.

If $G$ has all irreducible representations one dimensional then $G$ is an extension of a $p$-group (namely $G'$) by an abelian group as Ehud proved in his answer.

It still remains to show that this extension splits in order to deduce that $G$ is a semidirect product. Could you please explain why this extension should split?

I couldn't understand Pablo's the argument with the triangularizable matrices.