This is to record that Dylan's suggestion in the comments was on the money: Sections A.1 and A.4 of Higher algebra are precisely what is needed. Specifically there is an equivalence like this whenever $X$ is a topological space locally of singular shape, and $\mathrm{Shv}(X)$ is hypercomplete, e.g. if $X$ is a metric ANR locally of finite dimension.
Section A.1 defines in general what it means to have a locally constant sheaf valued in an $\infty$-category $C$ on a topological space $X$ (or more generally an $\infty$-topos). Now $D(X,R)$ is the $\infty$-category of hypercomplete $D(R)$-valued sheaves on $X$. The full subcategory of locally constant objects in Lurie's sense coincides precisely with the full subcategory of objects with locally constant cohomology. So if $\mathrm{Shv}(X)$ is hypercomplete then the $\infty$-category defined in my first bullet point is simply the category of locally constant $D(R)$-valued sheaves on $X$, as defined in Lurie.
What Lurie then explains/defines in A.4 is that for $X$ locally of singular shape, there is an equivalence of $\infty$-categories between locally constant $C$-valued sheaves on $X$, and functors $\mathrm{Sing}(X) \to C$. When $C=D(R)$ these are precisely parametrized $HR$-module spectra as in the second bullet point.