If you consider a frame of your bundle E, say $E_1$...$E_k$ then there will be forms $\omega_{jk}$ such that $$ \nabla E_j = \omega_{jk} E_k$$ (summation over repeated indices). The notation $\nabla_X E_j$ becomes $\omega_{jk}(X) E_k$. So for instance $$\nabla_X \nabla_Y (E_j) = X(\omega_{jk}(Y))E_k + \omega_{jk}(Y)\omega_{kl}(X) E_l $$ where I have applyied the rules you mentioned. On the other hand $$\nabla \circ \nabla (E_j) = \nabla(\omega_{jk} E_k) =d\omega_{jk}E_k - \omega_{jk}\wedge (\nabla E_k) = (d\omega_{jl} - \omega_{jk} \wedge \omega_{kl}) E_l, $$ (again applying the rules). Now you can apply the above two forms to vectors X and Y. You will need the formula $$ d\omega_{jl}(X,Y) = X(\omega_{jl}(Y)) - Y(\omega_{jl}(X)) - \omega_{jl}([X,Y]), $$ this is how the $\nabla_{[X,Y]}$ part comes out in the curvature. By applying all these ideas in the end you get the "usual" formula for the curvature. Reference: a good book on this is "From calculus to cohomology" by Madsen and Tornehave.