When you write $HH_*(A)$ I will assume that you mean the $R$-linear Hochschild homology of $A$.  (If you'd meant the absolute version, there would be no hope -- e.g., consider $A = R = \mathbb{Z}[x]/x^2$.)  

Then, under your hypothesis, it is easy to show that the Hochschild chain complex $HH^R_\bullet(A)$ is *perfect* over $R$.  I'll continue to use superscripts to denote the ground ring, and a bullet (rather than asterisk) to denote a chain model.

But you asked for "projective," so we must ask furthermore whether the homologies of this perfect complex are *flat*.  I believe that this is **not generally true** if $A$ is a dg-algebra.  I think it actually might be if $A$ is an ordinary algebra, but let me come back to that in an edit.


----------
**Producing a counter-example**

I believe that a counter-example, with $A$ a a smooth and proper *dg-*algebra, can be gotten in the following way:  

Follow Matt E's construction in http://math.stackexchange.com/questions/51146/hodge-number-jump-in-family-example to produce an example of a family over $W(k)$ where Hochschild-Kostant-Rosenberg and Hodge=>de Rham degeneration hold at the special fiber, and where the ranks of Hodge cohomology jump.  This ensures that the ranks of some Hochschild homology must also jump, violating flatness.


In more detail:

1. Take $X$ a smooth projective surface over the ring of Witt vectors $W(k)$ for $k$ an algebraically closed field of characteristic $p > 2$, whose geometric generic fibre has $p$-torsion in its middle dimensional (etale) cohomology.  [See the linked post for where to look for such an example.]  The linked discussion explains that the de Rham H^1 in the generic and special fiber have different rank. 

2. The example is now to consider the Hochschild homology *of $X$* (in the sense of the sheafy construction or of the dg-category $\mathrm{Perf}(X)$) over $R = W(k)$.   We can of course convert this to dg-algebra: Let $X \subset \mathbb{P}^N_R$ be a projective embedding, set $G$ to be the restriction of $\oplus_{i=0}^N O(-i)$, and set $A = RHom_X(G, G)$ as a (cofibrant) dg-$R$-algebra.  (The point is that $G$ generates $\mathrm{Perf}(X)$, so that $\mathrm{Perf}(A) = \mathrm{Perf}(X)$.) 

3. Then, $A$ will be smooth over $R$ because the dg-category $Perf(X)$ is smooth over $R$ (it'll come down to the structure sheaf of the relative diagonal being perfect on $X^2$).  And $A$ is perfect over $R$ since it is the pushforward of a perfect complex on $X$, and $X$ is proper flat over $R$.

3. Let $K$ denote the fraction field of $R$, and note that there are equivalences
$$ HH^R_\bullet(A) \stackrel{L}\otimes_R k = HH^k_\bullet(A \stackrel{L}\otimes_R k) = HH^k_\bullet(X_k) $$
$$ HH^R_\bullet(A) \otimes_R K = HH^K_\bullet(A \otimes_R K) = HH^K_\bullet(X_K) $$
Thus, to show that the homology modules are *not flat* over $R$ it is enough to show that the homologies of the two displayed right hand sides have different dimensions.  Notice that we had assumed that $p = \mathrm{char}(k) > 2 = \mathrm{dim} X_k$ so that HKR applies, the Hodge=>de Rham spectral sequences degenerate (we have a lifting to $W_2$!), and nothing is confusing.  Thus, the ranks on both sides are sums of Hodge cohomologies.  So:
$$ \dim_k H_1(HH^k_\bullet(X_k)) = \sum_{p-q=1} H^q(X_k, \Omega_{X_k}^p) = h^{1,0}(X_k) + h^{2,1}(X_k) = h^{1,0}(X_k) + h^{0,1}(X_k) = h^1_{dR}(X_k) $$
$$ \dim_K H_1(HH^K_\bullet(X_K)) = \sum_{p-q=1} H^q(X_K, \Omega_{X_K}^p) = h^{1,0}(X_K) + h^{2,1}(X_K) = h^{1,0}(X_K) + h^{0,1}(X_K) = h^1_{dR}(X_K) $$ 
And recall from 1 that these are different!