When you write $HH_*(A)$ I will assume that you mean the $R$-linear Hochschild homology of $A$. (If you'd meant the absolute version, there would be no hope -- e.g., consider $A = R = \mathbb{Z}[x]/x^2$.)
Then, under your hypothesis, it is easy to show that the Hochschild chain complex $HH^R_\bullet(A)$ is *perfect* over $R$. I'll continue to use superscripts to denote the ground ring, and a bullet (rather than asterisk) to denote a chain model.
But you asked for "projective." There are two things to say:
1. If $A$ is an ordinary algebra, then I will sketch an argument at the bottom for why this *is true*.
2. If $A$ is allowed to be a dg-algebra, then I believe that this is *not true* and I'll sketch how to get a counterexample.
----------
**Producing a counter-example (with $A$ a dg-algebra)**
I believe that a counter-example, with $A$ a a smooth and proper *dg-*algebra, can be gotten in the following way:
Follow Matt E's construction in https://math.stackexchange.com/questions/51146/hodge-number-jump-in-family-example to produce an example of a family over $W(k)$ where Hochschild-Kostant-Rosenberg and Hodge=>de Rham degeneration hold at the special fiber, and where the ranks of Hodge cohomology jump. This ensures that the ranks of some Hochschild homology must also jump, violating flatness.
In more detail:
1. Take $X$ a smooth projective surface over the ring of Witt vectors $W(k)$ for $k$ an algebraically closed field of characteristic $p > 2$, whose geometric generic fibre has $p$-torsion in its middle dimensional (etale) cohomology. [See the linked post for where to look for such an example.] The linked discussion explains that the de Rham H^1 in the generic and special fiber have different rank.
2. The example is now to consider the Hochschild homology *of $X$* (in the sense of the sheafy construction or of the dg-category $\mathrm{Perf}(X)$) over $R = W(k)$. We can of course convert this to dg-algebra: Let $X \subset \mathbb{P}^N_R$ be a projective embedding, set $G$ to be the restriction of $\oplus_{i=0}^N O(-i)$, and set $A = RHom_X(G, G)$ as a (cofibrant) dg-$R$-algebra. (The point is that $G$ generates $\mathrm{Perf}(X)$, so that $\mathrm{Perf}(A) = \mathrm{Perf}(X)$.)
3. Then, $A$ will be smooth over $R$ because the dg-category $Perf(X)$ is smooth over $R$ (it'll come down to the structure sheaf of the relative diagonal being perfect on $X^2$). And $A$ is perfect over $R$ since it is the pushforward of a perfect complex on $X$, and $X$ is proper flat over $R$.
3. Let $K$ denote the fraction field of $R$, and note that there are equivalences
$$ HH^R_\bullet(A) \stackrel{L}\otimes_R k = HH^k_\bullet(A \stackrel{L}\otimes_R k) = HH^k_\bullet(X_k) $$
$$ HH^R_\bullet(A) \otimes_R K = HH^K_\bullet(A \otimes_R K) = HH^K_\bullet(X_K) $$
Thus, to show that the homology modules are *not flat* over $R$ it is enough to show that the homologies of the two displayed right hand sides have different dimensions. Notice that we had assumed that $p = \mathrm{char}(k) > 2 = \mathrm{dim} X_k$ so that HKR applies, the Hodge=>de Rham spectral sequences degenerate (we have a lifting to $W_2$!), and nothing is confusing. Thus, the ranks on both sides are sums of Hodge cohomologies. So:
$$ \dim_k H_1(HH^k_\bullet(X_k)) = \sum_{p-q=1} H^q(X_k, \Omega_{X_k}^p) = h^{1,0}(X_k) + h^{2,1}(X_k) = h^{1,0}(X_k) + h^{0,1}(X_k) = h^1_{dR}(X_k) $$
$$ \dim_K H_1(HH^K_\bullet(X_K)) = \sum_{p-q=1} H^q(X_K, \Omega_{X_K}^p) = h^{1,0}(X_K) + h^{2,1}(X_K) = h^{1,0}(X_K) + h^{0,1}(X_K) = h^1_{dR}(X_K) $$
And recall from 1 that these are different!
----------
**The case where $A$ is discrete**
(I'll use the word "discrete" to indicate that something that was dg is actually concentrated in degree $0$.)
The argument is via three Lemmas:
**Lemma 1**: If $A$ is a discrete $R$-algebra (here $R$ is a discrete commutative ring), then $HH_\bullet^R(A)$ is a connective $R$-module. (Recall $M$ is connective if $H^i M = 0$ for $i>0$.)
*Pf*: Derived tensor products, over connective rings, preserve conective objects. $\square$
**Lemma 2**: If $A$ is smooth and proper over $R$, then there is a perfect pairing of perfect $R$-module
$$ HH_\bullet^R(A) \otimes HH_\bullet^R(A^{op}) \longrightarrow R $$.
*Sketch*: Smooth and proper $R$-algebras are fully dualizable objects in the Morita $\infty$-category of $R$-algebras (and perfect bi-modules), and $HH^R_\bullet(-)$ gives a symmetric monoidal functor from this Morita $\infty$-category to the $\infty$-category of complexes of $R$-modules localized at quasi-isomorphisms. When $A$ is smooth and proper, its dual in the Morita category is $A^{op}$ -- and thus the result follows! (Presumably similar discussion happens in Caldararu and Willerton's papers on the Mukai pairing.) $\square$
**Lemma 3**: Suppose that $P$ is a perfect $R$-module, and that both $P$ and its dual $RHom_R(P, R)$ are connective. Then, $P$ is a discrete projective $R$-module.
*Pf*: If $R$ is a field, this is clear. Thus, it follows that $P \stackrel{L}\otimes_R k$ is discrete for every map $R \to k$ from $R$ to a field. Now this is a standard criterion for a perfect complex to be a discrete projective module.$\square$