$\newcommand{\Gp}{\mathbf{Grp}}
\newcommand{\conj}{^{\mathbf{conj}}}
\newcommand{\Gpd}{\mathbf{Gpd}}
\newcommand{\Set}{\mathbf{Set}}
\newcommand{\ho}{\mathrm{ho}}
\newcommand{\Fun}{\mathrm{Fun}}
\newcommand{\Lan}{\mathrm{Lan}}
\newcommand{\colim}{\mathrm{colim}}
$
Yes. There is a "by-hands proof", where you just compute the colimits that are involved, but there turns out to be a nice proof if you go to $(2,1)$-categories. Namely, $\Gp\conj$ is equivalent to the homotopy category of a natural $(2,1)$-category: its objects are groups, its $1$-morphisms are group morphisms, and its $2$-morphisms are conjugations : if $f,g: G\to Q$ are morphisms, a $2$-morphism $f\implies g$ is an element $q\in Q$ such that $f^q = g$ (where $f^q : qf(-)q^{-1}$).
I'm going to call this $\Gpd$, and it's relatively easy to observe that $\Gp \simeq \Gpd_{1/}$ in a way that the functor $A$ can be identified with the composite $\Gp\simeq \Gpd_{1/}\to \Gpd\to \ho(\Gpd)\simeq \Gp\conj$.
Because $\Set$ is a $1$-category, restriction along $\Gpd\to\ho(\Gpd)$ induces an equivalence $\Fun(\ho(\Gpd),\Set)\to \Fun(\Gpd,\Set)$, so that left Kan extension is "silly": it's the observation that the functor factors through $\ho$.
Therefore, it suffices to prove that the left Kan extension of $\Gpd_{1/}\to \Set$ along $\Gpd_{1/}\to\Gpd$ is given by the set of conjugacy classes. But now we can reap the benefits of this approach: $\Gpd_{1/}\to\Gpd$ is a left fibration (as is $C_{x/}\to C$ for any $(\infty,1)$-category $C$ and object $x$), so that the left Kan extension of a functor $f:\Gpd_{1/}\to \Set$ along $p:\Gpd_{1/}\to \Gpd$ is given at $G$ by the colimit over the fiber of $f$ (note that $\Gp\to \Gp\conj$ is not a left fibration).
More precisely: $\Lan_p f(G) = \colim_{\alpha \in p^{-1}(G)} f(1\xrightarrow{\alpha} G)$. Here, $f(1\to G)$ is just the underlying set of $G$, and the $1$-groupoid $p^{-1}(G)$ is equivalent to $BG$, and it acts on $G=f(1\xrightarrow{\alpha} G) $ by conjugacy, so that this colimit is the quotient of $G$ by its conjugacy action.
So $\Lan_p X$ is the functor from $\Gpd$ to $\Set$ that sends $G$ to its set of conjugacy classes, and this factors through $\ho(\Gpd)\simeq \Gp\conj$ in the appropriate way.