As xuhan points out, SGA3 provides a fairly comprehensive view of what can be said about quotients involving group schemes. The price of this thoroughness is of course a lengthy technical treatment from which you might have trouble extracting just the amount of information you need. (SGA3 is currently available online here.) In any case, it would be helpful if xuhan expands the comment here into a full answer.

Two later expositions may be worth consulting for your specific questions, since they are written in different styles:

Demazure-Gabriel, Groupes algebraiques, Chap. III, section 3.3.

Jantzen, Representations of Algebraic Groups (2nd ed., AMS, 2003), I.5 (especially the early sections of this chapter). Note that Jantzen follows closely the Demazure-Gabriel treatment, but he tries to make his own version as self-contained as possible. Moreover, his study of group schemes is slanted particularly toward work with reductive group schemes over fields of prime characteristic.