There is a wide litterature for the classical Gauss sums. For $\chi$ a primitive Dirichlet character modulo $N$, it is given by $$\tau(\chi) = \sum_{n \text{ mod } N} \chi(n) \exp(2i\pi n/N).$$

An interesting fact is that primitive Dirichlet characters are essentially their own Fourier transform, up to this associated Gauss sum: $$\sum_{n \text{ mod } N} \chi(n) \exp(2i\pi nm/N) = \bar{\chi} (m) \tau(\chi). \qquad (\star)$$

I am interested in similar properties in general number fields $F$. Let $\omega$ be a nonzero element of $F$ and write $\mathfrak{a}$ for its denominator. Hecke defined some analogous Gauss sums by $$C(\omega) = \sum_{n \text{ mod } \mathfrak{a}} \exp(\mathrm{tr}(n^2 \omega)).$$

(here $n \text{ mod } \mathfrak{a}$ means that $n$ is an integer ideal in $\mathfrak{o}_F/\mathfrak{a}\mathfrak{o}_F$, where $\mathfrak{o}_F$ is the ring of integers of $F$). I have not found anything concerning the analogous Gauss sums as above, namely for a finite order character $\chi$ on integer ideals which is a character modulo $\mathfrak{a}$, $$\tau(\omega, \chi) = \sum_{n \text{ mod } \mathfrak{a}} \chi(n)\exp(\mathrm{tr}(n^2 \omega)).$$

Is there anything of this kind in the litterature? In particular, do we have an analogous result to $(\star)$?

ideal? You can find other treatments of Gauss sums over number fields and local fields in some articles in the 1977 Durham conference proceedings "Algebraic Number Fields" edited by Frohlich (not to be confused with the book by Cassels & Frohlich). See the contributions by Martinet on Artin $L$-functions and by Tate on local constants. $\endgroup$ – KConrad Nov 9 '18 at 3:32