Let $\mathcal{A}$ be a small category with some ( maybe no) colimits. What I would like to be able to do is add the rest of the colimits in a universal way. The Yoneda lemma will not work, since this simply adds all colimits formally. That is to say that you have new colimits that are different than the old. We do have maps from the newly created colimits to the old. The question is can we localize about these morphisms making the old and new colimits isomorphic?
Yes. More generally, let $A$ be a small category, and $D$ some set of "distinguished" colimits in $A$ — for example, you could take the set of all colimits that exist. A sheaf on $(A,D)$ is a functor $F: A^{\mathrm{op}} \to \mathrm{Set}$ such that for every colimit diagram $d\in D$, $Fd$ is a limit diagram in $\mathrm{Set}$. In particular, every representable functor $\hom(,a)$, for $a\in A$, is a sheaf, by definition of colimit. Denote by $\operatorname{Shv}(A,D)$ the full subcategory of $\operatorname{Fun}(A^{\mathrm{op}},\mathrm{Set})$ whose objects are sheaves (thus the morphisms are all natural transformations). Then we have a full faithful embedding of categories $\gamma: A \to \operatorname{Shv}(A,D)$ sending $a\mapsto \hom(,a)$. It should be obvious that the category $\operatorname{Shv}(A,D)$ is complete. Indeed, given any limit diagram in $\operatorname{Shv}(A,D)$, I claim that their limit in $\operatorname{Fun}(A^{\mathrm{op}},\mathrm{Set})$ actually lies in $\operatorname{Shv}(A,D)$, and this basically turns on the fact that "limits commute". It is less obvious, but also true, that $\operatorname{Shv}(A,D)$ is cocomplete. Also, there is a "sheafification" functor $\operatorname{Fun}(A^{\mathrm{op}},\mathrm{Set}) \to \operatorname{Shv}(A,D)$ which is adjoint to the forgetful map $\operatorname{Shv}(A,D) \to \operatorname{Fun}(A^{\mathrm{op}},\mathrm{Set})$. In fact, $\operatorname{Shv}(A,D)$ is universal in the following sense. Let $C$ be any cocomplete category, and $F : A \to C$ a functor such that for every distinguished diagram $d\in D$, $Fd$ is a colimit diagram in $C$. Then $F$ factors as $F = \tilde F \circ \gamma$, where $\gamma$ is the Yoneda embedding $A \to \operatorname{Shv}(A,D)$ from above, and $\tilde F : \operatorname{Shv}(A,D) \to C$ is a cocontinuous functor (preserves all colimits). Moreover, $\tilde F$ is unique up to unique isomorphism. The buzzwords for this construction are that $(A,D)$ is a sketch and that $\operatorname{Shv}(A,D)$ is a presentable category. The best reference I know for this material is Jiří Adámek and Jiří Rosický, Locally presentable and accessible categories, Cambridge University Press, 1994. We review some of this material (enough for some handson understanding) in the second section of my paper with Alex Chirvasitu — probably you only want to look at pp 6–13 or so, but maybe it is easier to find what you're looking for there than in the big Adámek and Rosický book. 

