Gerry and Emil in the [comments](https://mathoverflow.net/questions/339529/definition-of-the-gauss-symbol#comment849746_339529) are correct: it's the integer floor function.

Mastnak (citation below) considers Tahara's work in an appendix, "An excursion into group cohomology."  On p433 in Example A.6, working in the order $n$ cyclic group generated by $x$, Mastnak uses 

> ...the cocycle defined by $c_a(x^i, x^j) = a^{\lfloor \frac{i+j}{n}\rfloor}$ for $i,j \in \{0,1,\ldots,n-1\}$.

This is followed by 

> Remark. We can drop the condition $i,j \in \{0,1,\ldots,n-1\}$ in definition of $c_a$ if we replace $\lfloor \frac{i+j}{n}\rfloor$ by $\lfloor \frac{i+j}{n}\rfloor - \lfloor \frac{i}{n}\rfloor - \lfloor \frac{j}{n}\rfloor$.

For $i=k, j=-k$, suppose $k = an+b > 0$ with $a \in \Bbb{Z}$ and $b \in \{1,\ldots,n-1\}$ (if $b=0$ then $n|k$ contrary to your coprime condition, and the case $n=1$ doesn't make much sense in this context).  Then 
\begin{equation} \left\lfloor \frac{0}{n}\right\rfloor - \left\lfloor \frac{k}{n}\right\rfloor - \left\lfloor \frac{-k}{n}\right\rfloor = 0 - a - (-a-1) = 1.\end{equation}  

As to Tahara calling it the Gauss symbol, here's a passage from Wolfram MathWorld on the floor function and its evolving notation.

> Unfortunately, in many older and current works (e.g., Honsberger 1976, p. 30; Steinhaus 1999, p. 300; Shanks 1993; Ribenboim 1996; Hilbert and Cohn-Vossen 1999, p. 38; Hardy 1999, p. 18), the symbol $[x]$ is used instead of $\lfloor x \rfloor$ (Graham et al. 1994, p. 67). In fact, this notation harks back to Gauss in his third proof of quadratic reciprocity in 1808. However, because of the elegant symmetry of the floor function and ceiling function symbols $\left\lfloor x \right\rfloor$ and $\lceil x \rceil$, and because $[x]$ is such a useful symbol when interpreted as an Iverson bracket, the use of $[x]$ to denote the floor function should be deprecated.


Mitja Mastnak (2002), [Hopf algebra extensions arising from semi-direct products of groups](https://www.sciencedirect.com/science/article/pii/S0021869302991453), J. Alg. 251: 413--434 ([MSN](https://mathscinet.ams.org/mathscinet-getitem?mr=1900292)).