It's been a while since I've thought about this stuff so take this with a grain of salt. But if my memory serves correctly, it goes something like this:
We can define a canonical symplectic form on phase space in terms of the Lagrangian. The symplectic form gives us a bijective correspondence between tangent vector fields and 1-forms (it works the same as with Riemannian manifolds, the key is just that we have a perfect pairing on tangent spaces). Continuous symmetries are flows on phase space which preserve the symplectic structure. To a continuous symmetry, we can associate a tangent vector field (its derivative, sort of), and therefore a 1-form (via the symplectic form). This 1-form happens to be closed (I don't remember why, might edit this post later if I remember), and therefore we can get a scalar field by integrating it, at least locally. This scalar field is the conserved quantity.