I think the answer for a general Hermitian matrix $ A $ is no.

Let $ u, v $ be distinct eigenvectors of $ A $ with the same eigenvalue $ \lambda $ and normalized so that $ u^\ast u = v^\ast v = 1 $. Then $ \| u - v \| > 0 $ and
\begin{equation*}
| u^\ast Au - v^\ast A v | = | \lambda u^\ast u - \lambda v^\ast v | = | \lambda - \lambda | = 0.
\end{equation*}

A concrete counterexample is
\begin{equation*}
A = \left( \begin{array}{ccc} 2& & \\\\ & 2& \\\\ & & 1 \end{array} \right), u = \left( \begin{array}{c} 1 \\\\ 0 \\\\ 0 \end{array} \right), v = \left( \begin{array}{c} 0 \\\\ 1 \\\\ 0 \end{array} \right)
\end{equation*}

ADDED: if we assume all eigenvalues are distinct, then the above argument leads to some kind of bound. Again, $u,v$ are eigenvectors of $ A $ with corresponding eigenvalues $ \lambda_1, \lambda_2 $ and normalized with $ u^\ast u = v^\ast v = 1 $. Then
\begin{equation*}
\| u - v \| \leq 2
\end{equation*}
and
\begin{equation*}
| u^\ast Au - v^\ast A v | = | \lambda_1 - \lambda_2 |
\end{equation*}

so that if we choose
\begin{equation*}
\kappa(A) \geq \min_{\lambda_i, \lambda_j} \frac{2}{|\lambda_i - \lambda_j|}
\end{equation*}
we can guarantee your inequality will hold.