OK, I think I worked this out on my own; I'm leaving the answer here in case others need it.

Let me write $B_*(A)$ for the bar complex of $A$, and $B^*(A) = Hom(B_*(A), k)$ for the cobar complex. The latter is a differential (bi)graded algebra; its cohomology is $Ext_A^{*, *}(k, k)$. The quadratic dual $A^!$ is isomorphic to the diagonal part $\sum Ext_A^{n, n}(k, k)$.

In fact, there is a map $B^*(A) \to A^!$ of dgas (equipping $A^!$ with the trivial differential) which in cohomology induces the projection of the Ext algebra onto its diagonal part. This makes $A^!$ a module for $B^*(A)$. I claim that if we dualize the Koszul complex $K(A)$, we have

$$H^*(K(A)^*) \cong Tor_{B^*(A)}(k, A^!)$$

When $A$ is Koszul, the projection $B^*(A) \to A^!$ is a quasi-isomorphism (this is essentially one definition of Koszulity), so the right hand side is $k$, concentrated in degree $0$, recovering the fact that the Koszul complex is acyclic in this case.

We can see the claim as follows: Consider the bar resolution of $k$ over $A$; I may write this as the complex $(A \otimes B_*(A), d)$, where $d$ is the standard differential. Notice that we can split $d$ up as $d = d_1 + d_2$, where $d_2$ is the internal differential in $B_*(A)$, and $d_1$ mixes together the two tensor factors. We normally regard this as an $A$-resolution of $k$, free over $B_*(A)$. However, we can turn this on its head and regard it as a cofree resolution of $k$ as a $B_*(A)$-comodule.

Specifically, we map $k \to A \otimes B_*(A)$ by tensoring together the unit of $A$ with the unit of $B_*(A)$ (perhaps co-augmentation is a better term). Since the unit is primitive, this is a map of $B_*(A)$-comodules. We already know that this is an equivalence. Thus this is a cofree resolution of $k$ over the dg-coalgebra $B_*(A)$, coinduced over $A$.

Dualizing, we get a free resolution $(A^* \otimes B^*(A), d^*)$ of $k$ over $B^*(A)$. Then $Tor_{B^*(A)}(k, A^!)$ is the homology of the complex

$$[A^* \otimes B^*(A)] \otimes_{B^*(A)} A^! = A^* \otimes A^! = K(A)^*$$

The fact that the differential is the same as in the Koszul complex follows, for instance from Priddy's construction of the Koszul complex as a portion of the cobar resolution.