If we consider the usual formal construction of a path integral over fields with gauge symmetries e.g as in Weinbergs "The Quantum Theory of Fields - Volume 2" the notion of gauge invariance is clear in the sense that the formal quantities transform and behave in certain manners.
If we consider a mathematically sound path integral for a scalar non-gauge field on Euclidean spacetime, we find that an actual integral measure should live on a space of distributions, see e.g this version of the Osterwalder-Schrader theorem.
Now, suppose that we would have actually achieved a meaningful quantisation of, say, abelian Yang-Mills theory in a path-integral-like fashion. Then we could expect our path integral measure $\mu$ to live on the distribution space $\mathcal{S}^*(\mathbb{R}^4)^{4}$. But now the integration variable is a (four-component) distribution and no longer a smooth field. So how do we now phrase the requirement of gauge invariance?
Usually, we should have that a gauge transformation maps $\mathcal{G}_\chi : A_\mu \mapsto A_\mu + \partial_\mu \chi$ for suitable $\chi$, e.g Schwartz functions. But in the distributional setting, this looks rather restrictive. Should we then simply allow distributional $\chi$ and demand that $\mu \circ \mathcal{G}_\chi^{-1} = \mu$ for all gauge transformations? That looks like much too strong a requirement to me and I guess that any such measure would have to be trivial, i.e equal to zero.
There is however also a dual formulation one could consider, by demanding that the characteristic function should be invariant, i.e $\hat{\mu}(\mathcal{G}_\chi f) = \hat{\mu}(f)$ for all gauge transformations. Note that now $f$ is a four-component Schwartz function, so the usual gauge transformations with smooth $\chi$s are meaningful. While this looks very natural, I do not see how it connects to the common way of thinking about path integrals. In particular, there is then no way to dualise $\mathcal{G}_\chi$ back to the integration variable because the meaningful "adjoint" operator $\bar{\mathcal{G}}_\chi$ \begin{equation} \left( \bar{\mathcal{G}}_\chi A \right) \left( f \right) := A \left( \mathcal{G}_\chi f \right) \end{equation} cannot be linear. Even worse, it does not even map $S^*(\mathbb{R}^4)^4$ to $S^*(\mathbb{R}^4)^4$ such that $\mu \circ \bar{\mathcal{G}}_\chi^{-1} = \mu$ does not remotely make any sense.
How does one phrase gauge invariance in a mathematical measure-theoretical QFT setting?
