This is a special case of the general construction of cocompletions that preserve existing colimits. The general statement can be found as Theorem 6.23 of Kelly's *Basic Concepts of Enriched Category Theory*, and more explicitly as Proposition 11.4 and Theorem 11.5 of Fiore's *Enrichment and Representation Theorems for Categories of Domains and Continuous Functions* (in the case of small cocompletions, though the result is easily modified to work with a class of colimits instead). In summary, for classes $\Phi, \Psi$ of colimits for which the small category $\mathbf B$ is $\Phi$-cocomplete, there is a conservative $\Psi$-cocompletion $\widehat {\mathbf B}_\Phi$ of $\mathbf B$ preserving the $\Phi$-colimits. This means that the restriction of the (restricted) Yoneda embedding $\mathbf B \to \widehat {\mathbf B}_\Phi$ is $\Phi$-cocontinuous and exhibits a bijection between $\Phi$-cocontinuous functors $\mathbf B \to \mathbf C$ into cocomplete categories $\mathbf C$, and $\Phi$- and $\Psi$-cocontinuous functors $\widehat {\mathbf B}_\Phi \to \mathbf C$. Explicitly, $\widehat {\mathbf B}_\Phi$ is the subcategory of the category of presheaves on $\mathbf B$ which are $\Psi$-colimits of representables taking $\Phi$-cocones to limiting $\Phi$-cones.

In your setting, take $\Phi$ to be the class of colimits in the image of $\iota : \mathbf A \to \mathbf B$, and take $\Psi$ to be the class of finite colimits. Then $\widehat {\mathbf B}_\Phi$ is exactly the finite cocompletion of $\mathbf B$ relative to $\iota$.