It seems to me that the OP's last remarks about the difference of the cases when $\pi_1(M)$ is finite or infinite already give the answer to the question. Namely, the two cases are not that different, in that you either use that the universal covering $\tilde M$ satisfies the same hypotheses as $M$; or that the two points have infinitely many pre-images and by completeness of $\tilde M$ there are infinitely many geodesics joining them (that get projected down to the desired infinitely many geodesics on $M$). Maybe it can be more "didactic" to slightly reorganize proof as follows:

## Proof:

The crucial fact is that, from the Morse relations,

the number of geodesics joining two non-conjugate points $p$ and $q$ of index $m$ is greater or equal to the $m^{th}$ Betti number of the loop space of $M$ with coefficients in (any field) $\mathbf K$.

If $\pi_1(M)=0$ and $\dim M=n$, then $H_n(M;\mathbf K)=\mathbf K$ since $M$ is orientable and $H_i(M;\mathbf K)=0$, $i>n$. By Serre's result mentioned above, this implies that the singular homology of the loop space $\Omega_{x,x}(M)$ of $M$ with arbitrary base point $x\in M$ satisfies the following property: for any integer $i\geq 0$, there exists an integer $0< j< n$ such that $H_{i+j}(\Omega_{x,x}(M);\mathbf K)\neq0$. Thus $\Omega_{x,x}(M)$ has *infinitely many non zero Betti numbers*, hence there are *infinitely many geodesics* joining $p$ to $q$.

If $\pi_1(M)\neq0$, consider the universal covering $\tilde M$ of $M$ with the pull-back metric and notice it is complete and since $\tilde M\to M$ is a local isometry, points in the pre images of $p$ and $q$ are not conjugate to each other. Then if $\pi_1(M)$ is finite, $\tilde M$ is compact and hence satisfies the same hypotheses as $M$, so there are infinitely many geodesics joining any two pre images of $p$ and $q$ by $\tilde M\to M$, which clearly project to infinitely many geodesics joining $p$ and $q$. If $\pi_1(M)$ is infinite, there are infinitely many pre images of $q$ by $\tilde M\to M$, hence infinitely many geodesics joining them to a pre image of $p$, because $\tilde M$ is complete. These clearly project to infinitely many geodesics joining $p$ to $q$ on $M$.

## An interesting remark:

In a very similar way, one can prove that the number of geodesics between two distinct non-conjugate points in a non-necessarily compact, *but contractible*, manifold is either **odd or infinite**. (e.g., in $\mathbf R^n$ it is always odd).

*Proof.* If $M$ is contractible, also $\Omega_{p,p}(M)$ is contractible hence all Betti numbers of $\Omega_{p,p}(M)$ are 1. The Morse relations state that there exists a formal power series $Q$ so that $$\sum k_m\lambda^m=b_\lambda (\Omega_{p,p}(M),\mathbf K)+(1+\lambda)Q(\lambda), \quad \lambda\in\mathbf R$$ where $k_m$ is the number of geodesics between $p$ and $q$ of index $m$. Setting $\lambda=1$, we conclude that the total number of geodesics joining $p$ to $q$ is equal to $2Q(1)+1$, which is either odd or infinite.