There is a straightforward bound.
Consider A to be the $log(N)$$\log(N)$-fold tensor product of $H= \frac{1}{\sqrt{2}} \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}$.
A is Unitary and Hermitian. In fact A is just the Fourier transform.
Set $u$ to be all one vector, i.e. $u=(1, 1, 1, \ldots , 1)^T$.
And $v$ is all one vector with the first coordinate set to $-1$, i.e. $v=(-1, 1, 1, \ldots , 1)^T$
See $Au =(1, 0 , 0 , \ldots 0)^T$
and $Av$ has non zero entries in all coordinates.
Thus, $||sgn(Au)-sgn(Av)||_1 \geq N-1$$\|\operatorname{sgn}(Au)-\operatorname{sgn}(Av)\|_1 \geq N-1$.
Conclusion. No non-trivial bound.