The Schwartz kernel theorem seams relevant here. You might recall from your signal processing books that in the linear but non-time-invariant case we still get the output as a convolution of a kernel with integral $\int K(x,y) f(y)dy$ where $f$ is the input, but a different kernel for each point in time. The kernel theorem makes this rigorous as I recall where, $K$ then can be a distribution.
Once you have that theorem it is probably easy to get the statement you want. Indeed the kernel can hardly be time-varying if the system is to be time-invariant.
