Let $A$ be a symmetric linear operator of norm $\leq 1$ on the space of functions $f:S\to \mathbb{R}$, where $S$ is a set with $N$ elements. Define the inner product $\langle \cdot,\cdot\rangle$ by letting $\{x\}$ have measure $1/N$ for each $x\in S$.
Assume that, for every function $f$ such that $|f|_2=1$ and $|f|_\infty\leq K$, we have $|\langle f,A f\rangle|\leq \alpha$.
Does it follow that there must be a $Y\subset S$ of measure $O(1/K)$ (say) such that, for $X = S\setminus Y$ and any $f:S\to \mathbb{R}$ with $|f|_2=1$, $$|\langle f|_X, A(f|_X)\rangle|\leq 10 \alpha,$$ say?