Reformulate
Supposing $f$ is a solution to your first formulation, with $f(1) = \lambda$. Let $\tilde{f}(r) = \lambda^{-1} f(r)$. Then $\tilde{f}$ solves the differential equation
$$ \tag{1} r(r-2a) \tilde{f}'' + 2(r-a) \tilde{f}' - m(m+1) \tilde{f} = \frac{4a^2}{r(r-2a)} (\tilde{f} - 1) + \frac{4a(1-2a)}{(1-a)r(r-2a)} \tilde{C} $$
with
$$\tilde{f}'(1) = \tilde{C} \text{ and } \tilde{f}(1) = 1 \tag{2}$$
(Here $\tilde{C} = \lambda^{-1} C$.)
Hence your question is equivalent to asking
- whether there exists some parameter $\tilde{C}$ such that the initial value problem (1), (2) admits a solution with a finite limit
- whether said $\tilde{C}$ is unique
- and what is the finite limit.
For convenience of typing, I will drop all tilde signs below. Note that the $f$ I will refer to is therefore different from the $f$ in your original question.
Third question
Let's answer the third question first. Suppose for a moment that a solution exists with a non-zero limit $f_\infty = \lim f(r)$. Suppose first that $f_\infty > 0$ (the argument will be similar if $f_\infty < 0$.) Then there exists $R > 0$ such that for all $r > R$, we have
$$ r(r-2a) f'' + 2(r-a) f' > \frac{m^2}{2} f_\infty$$
and $f(r) > \frac12 f_\infty$. This shows that $f$ has no local maxima and hence must eventually be monotonic.
The same argument from my previous answer then shows that $\lim f'(r)$ exists (and hence must be zero) and that $|f'(r)|$ is at least size $1/r$ asymptotically. This contradicts the integrability of $f'(r)$.
This shows that if a solution were to exist with a finite $f_\infty$, then $f_\infty = 0$ necessarily.
Second question
Decompose $f = g + h_C$, where $g$ solves
$$ r(r-2a) g'' + 2(r-a) g' - m(m+1) g - \frac{4a^2}{r(r-2a)} g = - \frac{4a^2}{r(r-2a)} $$
with $g'(1) = 0$ and $g(1) = 1$, and $h_C$ solves
$$ r(r-2a) h'' + 2(r-a) h' - m(m+1) h - \frac{4a^2}{r(r-2a)} h = \frac{4a(1-2a)}{(1-a)r(r-2a)} C$$
with $h'(1) = C$ and $h(1) = 0$.
We see that the mapping $C \mapsto h_C$ is linear. So let's concentrate on the case $C = 1$.
Then $m(m+1) h + \frac{4a^2}{r(r-2a)} h + \frac{4a(1-2a)}{(1-a)r(r-2a)} > 0$ whenever $h \geq 0$. So $h$ cannot have a positive local maximum. Since $h$ is initially increasing and $h(1) = 0$, this means that $h$ is strictly increasing. And hence $\lim h(r)$ exists in $(0,\infty]$ by the monotone convergence theorem; in particular, $\lim h(r) \neq 0$.
This answers question 2. Suppose with the value $C_0$, we find $f$ a solution to the original problem. If $D \neq 0$, then the solution $\hat{f}$ using the value $C_0 + D$ will have limit $\lim \hat{f}(r) = \lim f(r) + \lim h_D(r) \neq 0$. And hence $C_0$ is unique. Furthermore, since we established that there can be no finite limit besides $0$, this means that $\lim h_D$ is necessarily $\pm \infty$.
First Question (Partial answer)
I will use the functions $g$ and $h_C$ as the previous part.
The equation satisfied by $g$ can be written as
$$ [(r^2 - 2ra)g']' = m(m+1) g + (g-1) \frac{4a^2}{r(r-2a)} $$
Note that if $g(r) \geq 1$ then the RHS is positive. And once $g'$ turns positive, then $g$ is increasing. So it is not too hard to argue in fact that $(r^2-2ra)g'$ is strictly increasing on $[1,\infty)$, and hence $g$ is also monotonically increasing.
As we noted before, $h_1$ (and hence $h_C$ for any $C > 0$) is also monotonically increasing.
- First, I claim that for any $C > 0$ we have that $h_C$ and $g_C$ intersects at most once. Suppose the two curves intersect the first time at $r_0$. Then since $g(1) > h_C(1)$ we have that $h'_C(r_0) \geq g'(r_0)$. But their difference satisfies the equation
$$ [ r(r-2a) (h'_C - g')]' = [ m(m+1) + \frac{4a^2}{r(r-2a)} ] (h_C - g) + \frac{4a(1-2a)}{(1-a)r(r-2a)} C + \frac{4a^2}{r(r-2a)} $$
which shows that after $r_0$ the value $h'_C - g'$ will be always positive.
- For $r > 1$, let $C(r)$ be the unique value such that $h_{C(r)}(r) = g(r)$; in other words let $C(r) = \frac{g(r)}{h_1(r)}$. The analysis in step 1 indicates that $C(r)$ is a decreasing function of $r$.
- By the monotone convergence theorem $\lim C(r)$ converges to a finite, non-negative limit $C_\infty$.
I believe that $f = g - h_{C_\infty}$ is the desired solution.
- It is clear from construction that $g - h_{C_\infty}$ is everywhere strictly positive. And hence for any $C < C_\infty$, we have that $\lim_{r\to\infty} g - h_{C} = +\infty$, using that $\lim h_1 = +\infty$.
- It is also clear from the construction that $\lim_{r\to\infty} g - h_C = -\infty$ for any $C > C_\infty$.
- So if there exists a solution, it must correspond to $C_\infty$.
I don't have the time to work out all the details, so here's a sketch for what needs to be done to prove that $C_\infty$ works.
- Suppose that at some point $r$ you have that $f(r) > \frac{4a}{m(m+1)r(r-2a)}[ a + \frac{1-2a}{1-a} C_\infty]$. and $f'(r) > 0$. Then for some sufficiently small $\epsilon$ we have $f - h_\epsilon$ is still everywhere positive. Here we use that we can choose $\epsilon$ small enough and take advantage of the strict positivity of $f$ on $[1,r]$, and that on the interval $[r,\infty)$ the initial conditions $f(r) - h_\epsilon(r) > \frac{4a}{m(m+1)r(r-2a)}[ a + \frac{1-2a}{1-a} (C_\infty + \epsilon) ]$ and $f'(r) - h_\epsilon'(r) > 0$ implies that $f - h_\epsilon$ is strictly increasing beyond that point. But this would contradict the definition of $C_\infty$.
- So for every $r$ we must have at least one of $f'(r) \leq 0$ or $f(r) < \frac{4a}{m(m+1)r(r-2a)}[ a + \frac{1-2a}{1-a} C_\infty]$ holds. This can be seen to imply that the larger of $f(r)$ and the decaying envelope (RHS of the inequality for $f(r)$) is a decreasing function. By the monotone convergence theorem this has a limit.
- If the limit is non-zero, this means $f$ is eventually larger than the envelope, and hence the limit is the limit of $f$. But this contradicts the answer to Question 3. And hence the limit is 0. By comparison this shows that this is also the limit of the positive function $f$.
