$\newcommand{\R}{\mathbb R}$The answer is NO. I will provide below a counterexample in dimension $d=1$.

**Preliminaries:**
Let's agree that the entropy is
$$
H(\rho)=\int_{\mathbb R}\rho(x)\log\rho(x) dx
$$
whenever $\rho=\rho(x) dx$ is absolutely continuous w.r.t. the Lebesgue measure $dx$. The counterexample to full (weak, or Wasserstein) continuity of the entropy is the typical "binary 1-0-1-0..." oscillations. I guess this is clear if you come from an information-theory background, I actually learnt this myself from here on Math Overflow. To be more precise, let
$$
A_n=\bigcup_{i=0}^{n-1}\Big[2i/(2n)\,,\,(2i+1)/2n\Big)
,\qquad
\rho_n=2\chi_{A_n}(x) dx
$$
and
$$
A=[0,1]\qquad\rho_*=\chi_{[0,1]}(x)dx.
$$
You can think of this as a sequence of binary words spread across the fixed domain $x\in[0,1]$, the value $\rho_n(x)=2$ corresponding to the $1$-bit and the value $\rho_n(x)=0$ to the $0$-bit.
So, we have a sequence of words $\rho_1=(1,0)$, then $\rho_2=(1,0,1,0)$, and so on, $\rho_n=\underbrace{(1,0,\dots,1,0)}_{n\text{ times}}$, each time with thinner and thinner elementary bits of width $\frac 1{2n}$.
It is easy to check that $\rho_n$ converges narrowly (in duality with $C([0,1])$ test functions) to $\rho_*$, hence also in the $W_2$ distance.
(Recall that, in bounded domains, convergence in any Wasserstein distance is equivalent to narrow convergence).
However, one can compute explicitly
$$
H(\rho_n)=n\times 2\log 2\times\frac 1{2n}=\log 2
$$
(number $n$ of bits, each of height $2$ and of width $\frac1{2n}$).
This obviously does not converge to $H(\rho_*)=\int 1\log 1\,dx =0$.

The counterexample below will be constructed by suitably interpolating the $\rho_n$'s in time.

**Construction of the counterexample:**
Pick any decreasing sequence $t_k\to 0$ with $t_1=1$, and set $h_k=t_k-t_{k+1}$.
By an immediate extraction argument, there exists a subsequence $\rho_k=\rho_{n_k}$ such that
$$
W_2(\rho_k,\rho_*)\leq h_k
$$
and therefore by triangular inequality
$$
W_2(\rho_k,\rho_{k+1})\leq h_k+h_{k+1}
$$
for all $k\geq 1$.
Let us now define the curve $(\rho(t))_{t\in(0,1]}$ by setting
$$
t=t_k:\qquad
\rho(t_k):=\rho_k
$$
and
$$
t\in(t_{k+1},t_k):\qquad
\rho(t):=\text{the time-}h_k\text{ geodesic }W_2 \text{interpolation between }\rho_{k+1},\rho_k.
$$
(I hope I don't need to make this more explicit.)
By construction this curve is absolutely continuous (at least for $t>0$) with piecewise-constant metric speed
$$
t\in(t_{k+1},t_k):\qquad
|\dot \rho|(t) =\frac{W_2(\rho_k,\rho_{k+1})}{h_k}
\leq \frac{h_k+h_{k+1}}{h_{k+1}}.
$$
Moreover one also has that, for any small $t_0>0$ and with the appropiate choice of $k_0$ such that $t_0\in[t_{k_0},t_{k_0+1})$,
$$
\int_{t_0}^1|\dot\rho|(t)dt
\leq \sum\limits_{k=1}^{k_0}\frac{h_k+h_{k+1}}{h_k} h_k
\leq \sum\limits_{k\geq 1}h_k+h_{k+1} \leq 2,
$$
since by construction $\sum h_k=1$. This shows that the metric speed $t\mapsto |\dot\rho|(t)$ is globally $L^1$ up to $t_0=0^+$. By standard completeness one can extend by continuity $\rho(0)=\lim \rho(t_k)=\rho_*$ while maintaining absolute continuity (including up to $t=0$), and the thesis follows since then
$$
H(\rho(t_k))=\log 2
\qquad \text{but}\qquad
H(\rho(0))=0.
$$