I'll have more time to write and provide a more thorough answer later, but I think the most straightforward proof (which I agree is hard to find) comes via sheaf theory: On the one hand, there is a sheaf of locally finite singular chains whose hypercohomology is your $H^{lf}$. I work out the details in the setting of intersection homology on pseudomanifolds in Section 3 of the following paper, but the ordinary (non-intersection) lf homology on locally compact (paracompact?) spaces is essentially a special case via basically the same arguments:
"Singular chain intersection homology for traditional and super-perversities"
Transactions of the American Mathematical Society 359 (2007), 1977-2019
On the other hand, you can find a good treatment of the dualizing complex in Borel's book on intersection homology, section V.7. The idea then is to show that these two sheaves are quasi-isomorphic. This is the part I'll have to think about and add latter, though I'll note that it's not so hard for manifolds: in that case, it's not so hard to show by hand that both have to be quasi-isomorphic to the orientation sheaf (shifted appropriately). That's shown in Borel for the dualizing complex, and follows for the sheaf of locally finite singular chains via the local homology computation on the manifold.
Sorry I don't have time to write more now, but hopefully that's a start on the ideas.
Added later:
Oookay, this was harder than I thought. I’m not sure I can give you a really satisfying answer, but I think I can convince you that what you want is contained in Bredon as Corollary V.12.21. This corollary pretty much says that if $X$ is nice enough (though still pretty general), then ${}_SH_p^{\Phi}(X;\mathcal B)\cong H_p^{\Phi}(X;\mathcal B)$, where here $\mathcal B$ is a sheaf and $\Phi$ is a paracompactifying family of supports. For what we want, let’s take $\Phi$ to be all the closed sets, so we need to assume $X$ paracompact. We’re also going to let $\mathcal B$ be the constant sheaf with stalk the ground ring $L$. Let’s call that $\mathcal R$ (since $\mathcal L$ is used for lots of other things in these sections of Bredon). In all of the definitions we’ll talk about, $\mathcal B$ would come in as a tensor product factor, so letting it be constant basically lets us ignore those factors.
Now, on the left we have ${}_SH_p^{cld}(X;\mathcal R)$, which is Bredon’s singular homology with (now constant) sheaf coefficients. He gives a pretty good explanation for this back on pages 287-289. The key point connecting this to what I wrote previously is the following. Suppose you start with the (homologically graded) presheaf he calls $\Delta_*^c(X,X-\bar U;R)$, which is just the usual relative singular chain complex. Since our spaces are locally compact, this sheafifies to the same sheaf as $\Delta_*^{lf}(X,X-\bar U;R)$, the sheaf of locally finite singular chains, whose hypercohomology (or hyperhomology with this indexing) computes the locally finite singular homology (more details about this can be found elsewhere in Bredon or in the Transactions paper of mine that I already mentioned). Though actually here you don’t even need the hypercohomology; on the top of page 289 he just takes the homology of the complex of sections (which is the same as the hyperhomology because the sheaf is homotopically fine – shown earlier in the book). The argument on page 288-289 further shows that the resulting homology groups are isomorphic to singular homology as he defined it on page 288, using basically the same sheaf but also employing the direct limit over subdivisions. So, the upshot is that the term on the left is locally finite singular homology.
For the term on the right, we need to trace through the definitions starting on page 292. By the boxes at the top, $$H_p^{cld}(X;\mathcal R)= H_p(C_*^{cld}(X;\mathcal R))= H_p(\Gamma(\mathcal C_*(X;\mathcal R))) =H_p(\Gamma(\mathcal D_*(\mathcal J^*(X;L)))\otimes \mathcal R),$$ where $L$ is our ground ring. So if $X$ is locally connected, we can ignore the $\mathcal R$ and we’re just taking the homology of the global sections of $\mathcal D_*(\mathcal J^*(X;L))$. I claim this is just the Verdier dualizing sheaf with homological indexing. To see this, we unwrap more definitions! By the top of page 292, $\mathcal J^*(X,L)$ is just the canonical injective resolution of the constant sheaf (he also calls the constant sheaf $L$ and I have called it $\mathcal R)$. Let’s just call this injective (and hence also c-soft) resolution $\mathcal K^*$. So we need to understand $\mathcal D(\mathcal K^*)$. By the definitions on pages 289-290, $\mathcal D$ should take a second argument $M$, but there’s a note on page 290 saying it’s omitted from the notation if $M=L$. So we’re really after $\mathcal D(\mathcal K^*;L)$.
By the second box on page 290, $\mathcal D(\mathcal K^*;L)$ means $\mathcal D(\Gamma_c\mathcal K^*;L)$, where $\Gamma_c\mathcal K^*$ is the cosheaf given by $(\Gamma_c\mathcal K^*)(U)=\Gamma_c(\mathcal K^*|U)$ by the definition part of Proposition V.1.6. But now piecing together the meaning of Definition V.2.1, which is given explicitly on the top of page 291, $\mathcal D(\Gamma_c\mathcal K^*;L)(U)=Hom(\Gamma_c\mathcal K^*(U),L^*)$, where $L^*$ is an injective resolution of $L$ according to the beginning of Section V.2. So, altogether, $\mathcal D(\Gamma_c\mathcal K^*;L)$ is just the sheaf $U\to Hom(\Gamma_c(\mathcal K^*|U),L^*)$.
But this is exactly the definition of the dualizing sheaf given by Borel on page 119 of Intersection Cohomology. That this is $f^!$ of the constant sheaf on a point is on Borel page 131.
So there you go. Tracing through the Bredon definitions is a nightmare, and not a satisfying conceptual thing at all, but it’s a proof.
To say just a couple of words about the proof of the Corollary, the point is that he defines at some point the idea of a ``flabby quasi-n-coresolution of L,’’ where here $L$ is the notation for the constant sheaf again. The point of these is that if you have one, say $\mathcal L_*$, and want to compute $H_*(X;\mathcal B)$ for some sheaf $\mathcal B$, then instead of going through the horrible definitions we just unwound, you have $H_*(X;\mathcal B)$ is just $H_*(\Gamma(\mathcal L_*\otimes \mathcal B))$. That’s Theorem V.12.20, and this is sort of like what happens in Borel V.6.7 (or other places) when you make a sheaf complex c-soft by tensoring it with a flat c-soft resolution of the constant sheaf. But a bit earlier in Theorem V.12.14 it’s shown that the same sheaf used to compute singular homology (the one using the subdivisions, called $\mathfrak S_*$) is such a flabby quasi-n-coresolution of $L$. That actually just follows from the definitions and the properties on the space. So it follows that $H_*(X;\mathcal B)\cong H_*(\Gamma(\mathfrak S_*) \otimes \mathcal B))$, which is the definition of singular homology with coefficients in $\mathcal B$.
It’s really a shame this isn’t all brought out more clearly in Bredon’s exposition.
Whew.
