I'm not sure I have much to add beyond my comment, but I might add a point of view (which could in turn provide some search terms).

I'd situate this construction within the bicategory of (small) categories, profunctors/bimodules, and transformations between them. Recall that a profunctor $R: C \nrightarrow D$ is a functor of the form $C^{op} \times D \to Set$ (conventions may differ), composed much in the way relations are, via the formula

$$(C \stackrel{R}{\nrightarrow} D \stackrel{S}{\nrightarrow} E)(c, e) = \int^{d: D} R(c, d) \times S(d, e).$$

(If you like, you can consider the bicategory of profunctors as biequivalent to a *strict* 2-category whose objects are small categories and whose morphisms $C \to D$ are given by cocontinuous functors $Set^C \to Set^D$.)

This bicategory is compact closed in an evident bicategorical sense: we have a symmetric monoidal bicategory whose tensor at the object level is given by cartesian product of small categories, and each object $C$ has a monoidal dual given by the opposite category $C^{op}$. For each $C$, the unit $\eta_C: 1 \nrightarrow C^{op} \times C$ is given by $\hom_C: C^{op} \times C \to Set$. (In the cocontinuous functor picture, it's the unique (up to isomorphism) cocontinuous functor $Set \to Set^{C^{op} \times C}$ that takes the terminal object $1$ to $\hom_C$.) The counit $\epsilon_C: C \times C^{op} \nrightarrow 1$ may also be described by a hom-functor, but it is probably more illuminating to think of it in terms of the cocontinuous functor picture, given by taking the coend $\int^C: Set^{C^{op} \times C} \to Set$.

Since we are working in a compact closed (bi)category, we can expect certain resonances with constructions in other compact closed categories, such as the category of finite-dimensional vector spaces. The construction in question is a profunctor composite

$$1 \stackrel{\eta_C}{\nrightarrow} C^{op} \times C \stackrel{F^{op} \times G}{\nrightarrow} D^{op} \times D \stackrel{\epsilon_{C^{op}}}{\nrightarrow} 1$$

which is certainly akin to trace operations in linear algebra. Thus, in linear algebra over a field $k$, we have the notion of trace of an endomorphism $f: V \to V$, which we can form categorically as the composite:

$$\text{Tr}(f) = \left(k \stackrel{\eta_V}{\to} V^\ast \otimes V \stackrel{1 \otimes f}{\to} V^\ast \otimes V \stackrel{eval_V}{\to} k \right)$$

where the first map $\eta_V$ takes $1 \in k$ to $\sum_{i = 1}^n f^i \otimes e_i$ (here $\{e_1, \ldots, e_n\}$ is a basis of $V$ and $f^i$ is the dual basis; the expression $\sum_{i=1}^n f^i \otimes e_i$ is independent of basis). Similarly, we speak of the trace of an endoprofunctor $B: C \nrightarrow C$; after a brief Yoneda-lemma type calculation, one finds that the composite

$$1 \stackrel{\eta_C}{\nrightarrow} C^{op} \times C \stackrel{1 \otimes B}{\nrightarrow} C^{op} \times C \stackrel{\epsilon_{C^{op}}}{\nrightarrow} 1$$

is the profunctor $1 = 1^{op} \times 1 \to Set$ taking the unique object of $1$ to $\text{Tr}(B) = \int^{c: C} B(c, c)$.

Thus we could also describe your construction as the trace of the endoprofunctor or endobimodule $B: C \nrightarrow C$ defined by $B(c, d) = \hom_D(Fc, Gd)$. Possibly this gives a useful search term.

Usually such traces are challenging to calculate explicitly (for example, determining the trace of an identity functor can be nontrivial). Among the properties of trace formally deducible from compact closed structure is $\text{Tr}(B \circ B') \cong \text{Tr}(B' \circ B)$.