Almost squared finite groups

Question

Definition. A finite group $G$ is called squared (resp. almost squared) if there exists a subset $A\subseteq G$ such that $G=\{ab:a,b\in A\}$ and $|G|=|A|^2$ (resp. $|G|=|A|^2-1$). Such a set $A$ will be called an (almost) square root of $G$.

According to the answers to this MO-post, no nontrivial finite group is squared.

In contrast, nontrivial almost squared finite groups do exist. The simplest one is the 3-element cyclic group $C_3$. Any 2-element subset of $C_3$ is an almost square root of $C_3$.

A less trivial example is the dihedral group $D_8=\langle a,b\;|\;a^4=b^2=1,\;bab=a^3\rangle$ with almost square root $A=\{a,b,ba\}$.

Three other examples of almost squared groups (found by GAP) are:

$\bullet$ the symmetric group $S_4$;

$\bullet$ the general linear group $GL(2,3)$ of non-degenerate $2\times 2$ matrices over the 3-element field,

$\bullet$ the symmetric group $S_5$.

Problem 1. Find more examples of almost squared finite group. Are there infinitely many almost squared finite groups?

Remark. Using GAP, Voldymyr Gavrylkiv established that among groups of order $<168$ the only almost squared groups are the groups $C_3$, $D_8$, $S_4$, $GL(2,3)$, and $S_5$. Those groups have orders 3, 8, 24, 48, and 120, respectively. It is interesting that no almost squared group of order 80 exists.

Problem 2. What can be said about the structure of almost squared groups?

Remark. Alex Ravsky observed that for an almost square root $A$ of an almost squared group $G$, the center $Z(G)$ of $G$ is almost contained in the set $A^2=\{a^2:a\in A\}$ in the sense that $Z(G)\setminus A^2$ contains at most one element. So, $|Z(G)|\le|A|+1=1+\sqrt{|G|+1}$, which implies that the unique almost squared commutative group is $C_3$.

The only known (at the moment) almost squared noncommutative groups $D_8$, $S_4$ and $GL(2,3)$ have even cardinality.

Problem 3. Is the cardinality of any almost squared noncommutative group even?

Remark. It can be shown that all noncommutative groups of odd order $<675=3^3\times 5^2$ are not almost squared.

Problem 3'. Is there an almost squared group among groups of order $675$?

Problem 4. Let $A$ be an almost square root of an almost squared non-commutative finite group $G$. Are there distinct elements $a,b\in A$ such that $a^2=b^2=g$ for some $g\in G$ such that:
$\bullet$ $g\in Z(G)$?
$\bullet$ $g^2\in Z(G)$?
$\bullet$ $g^2=1$?
$\bullet$ $g=1$?
$\bullet$ $g$ has order $\le 3$?

Remark. For any almost square root in any of three known almost squared noncommutative groups $D_8,S_4,GL(2,3)$, there exist distinct elements $a,b\in A$ such that $a^2=b^2=1$. In these group the center contains at most two elements.

The following problem was suggested by @LSpice in his comment.

Problem 5. Let $G$ be an almost squared noncommutative group. Is $z^2=1$ for any central element of $G$?

Answer 1

Some preliminary comments. I may add more later: Let $G$ be any finite group. If $S$ is a subset of $G$, let $S^{+} = \sum_{s \in S} s$ in the group algebra $\mathbb{C}G$. Suppose that $G$ has an almost square root $A$. If $1 \in A$, then $1 a = a 1 $ for every $a \in A$, so that $|A| = 1 $ and $|G| = 3 $, so suppose from now on that $1 \not \in A$. We have $(A^{+})^{2} = x + G^{+}$ for some $x \in G.$ Notice that $A^{+}$ commutes with $G^{+}$, so that $A^{+}$ commutes with $x$. Hence $xAx^{-1} = A$, so that the set $A$ is invariant under conjugation by $x$. Also we have $(x^{n}A^{+})^{2} = x^{2n+1} + G^{+}$ for every integer $n$, so that $x^{n}A$ is another almost square root for $G$. If the order of $x$ is not a power of $2$, then there is an integer $n$ such that the order of $x^{2n+1}$ is a power of $2$. Hence if $G$ has an almost square root, then we may arrange matters so that the order of $x$ is a power of $2$ (allowing the possibility of order $1$). In particular, we may arrange matters so that $x = 1$ when $G$ has odd order.

Now suppose that $x$ has $2$-power order greater than one. Let $a,b \in A$ with $x = ab.$ If $a,b \not \in C_{G}(x)$ then $x = a^{x}b^{x}$ is the only other factorization of $x$ as a product of two elements of $A$. If $a,b \in C_{G}(x)$ with $a \neq b$ then $x = ab = ba$ are the only expressions of $x$ as a product of two elements of $A$. If $a = b \in C_{G}(x)$ then there is exactly one more expression $c^{2} = x$ with $c \in A$ (since $x$ can't then be a product of two elements of $A$, each outside $C_{G}(x)).$

Now let $H = C_{G}(x)$ and $D = A \cap H.$ Suppose that $ d \in H $ with $d \neq x.$ Then $d = uv$ for some $u,v \in A$ and $u,v$ are unique. Then $d = d^{x} = u^{x}v^{x}$ and $u^{x},v^{x} \in A$. Hence $u = u^{x}$ and $v = v^{x}.$ Thus $u, v \in D$.

Now we have $(D^{+})^{2} = H^{+} \pm x $. If the product is $H^{+} + x,$ then $D$ is an almost square root for $H = C_{G}(x).$

Suppose then $(D^{+})^{2} = H^{+}-x.$ Note that $H$ has even order as $x \in H$. In the regular representation of $H$, the element $H^{+}- x$ has the eigenvalue $|H| - 1$ with multiplicity $1$, and $D^{+}$ has the eigenvector $|D|$ on that eiegenspace. Hence $|H|-1$ is the square of an odd integer. Thus $|H| \equiv 2$ (mod $8$). In that case, $\langle x \rangle$ is a Sylow $2$ subgroup of $H = C_{G}(x) = N_{G}(\langle x \rangle)$, so is also a Sylow $2$-subgroup of $G$. This is a standard group-theoretic argument, but I explain it for non-group theorists : Note that $x$ lies in some Sylow $2$-subgroup $S$ of $G$. Suppose that $|S| > 2.$ If $x \in Z(S)$, then $S \leq C_{G}(x),$ so that $|S|$ divides $|H|$ which is not the case. If $x \not \in Z(S)$, then $\langle x, Z(S) \rangle \leq H$ and $|\langle x, Z(S) \rangle | \geq 2|\langle x \rangle |,$ a contradiction.

However, we can now reach a contradiction. If $G$ is a group of even order such that $A$ is an almost square root for $G$, then $|A|^{2} = |G| + 1$, so that $|G|+1$ is the square of an odd integer. Hence $|G| + 1 \equiv 1$ (mod $8$), so that $|G|$ is divisible by $8$.

But in the present case, $|G|$ is not divisble by $4$. Hence the situation $(D^{+})^{2} = H^{+} - x$ can not occur when $x$ is a non-identity element of $2$-power order of $G$, and $G$ has an almost square root $A$ such that $(A^{+})^{2} = G^{+} + x$, where $H = C_{G}(x).$

In particular, we may deduce that if $G$ is a finite group of even order with an almost square root $A$ such that $(A^{+})^{2} = G^{+} + x$ for some non-identity element $x$ whose order is a power of $2$, then $D = A \cap C_{G}(x)$ is an almost square root for $H = C_{G}(x).$

To recap : if $G$ is a finite group with an almost square root $A$, and we have $(A^{+})^{2} = G^{+} + x$ for some non-identity element $x$, then we can find an integer $n$ such that $x^{n}A$ is an almost square root for $G$ with $(x^{n}A^{+})^{2} = G^{+} + x^{2n+1}$, where the order of $x^{2n+1}$ is a power of $2$ (possibly $1$). This reduces to considering the case that the order of $x$ is a power of $2$ (possibly $1$). If the order of $x$ is a power of $2$ greater than one, the group $C_{G}(x)$ has the almost square root $D = A \cap C_{G}(x).$

Hence a key case to understand is when $x$ is a central element of $G$ whose order is a power of $2$ (possibly $1$).

Continued : Notice also that if $x$ has even order, then $|C_{G}(x)|$ is a group of even order with an almost square root, so that $|C_{G}(x)|$ has order divisible by $8$.

Answer 2

The following theorem yields a partial answer to Problem 2.

A subset $C$ of a group $G$ is called unfree if $xy=yx$ or $x^2=y^2$ for any elements $x,y\in C$.

For a group $G$ let $ucov(G)$ be the smallest cardinality of a cover of $G$ by unfree subsets of $G$.

Theorem 1. If $A$ is an almost square root of an almost squared finite group $G$, then $$\sqrt{|G|+1}=|A|\le 1+ucov(G).$$

Proof. Let $A$ be an almost square root of $G$ and $\mathcal U$ be a cover of the group $G$ by unfree subsets such that $|\mathcal U|=ucov(G)$. We lose no generality assuming that the cover $\mathcal U$ consists of pairwise distinct sets.

Claim. There exists an element $a\in A$ such that for any distinct elements $x,y\in A\setminus\{a\}$ we have $xy\ne yx$ and $x^2\ne y^2$.

Proof. To derive a contradiction, assume that for any $a\in A$ there exist elements $x,y\in A\setminus\{a\}$ such that $xy=yx$ or $x^2=y^2$. For every $g\in G$ consider the set $P_g=\{(a,b)\in A\times A:ab=g\}$. By the definition of an almost square root, there exists a unique element $g\in G$ such that $|P_g|=2$ and $|P_x|=1$ for all $x\in G\setminus\{g\}$. Take any pair $(a,b)\in P_g$. By our assumption, there exist two distinct elements $x,y\in A\setminus\{a\}$ such that $xy=yx$ or $x^2=y^2$. It follows that the set $P_g$ coincides with $\{(x,y),(y,x)\}$ or $\{(x,x),(y,y)\}$, which implies that $a\in\{x,y\}$. But this contradicts the choice of $x,y$. $\quad\square$

Claim implies that for any $U\in\mathcal U$ we have $|U\cap (A\setminus\{a\})|\le 1$. Consequently, $$|A|=1+|A\setminus\{a\}|\le 1+|\mathcal U|=1+ucov(G).\quad\square$$

Now it remains to find some upper bounds on the number $ucov(G)$.

For a group $G$ let $ccov(G)$ be the smallest cardinality of a cover of $G$ by cyclic subgroups. Since each cyclic group is an unfree set, we get the upper bound $ucov(G)\le ccov(G)$. Also $ccov(G)\le|G|-1$ for any nontrivial group $G$ and $ccov(G)\le|G|-2$ for any group which is not Boolean.

Lemma 1. $ucov(G)\le ccov(G/Z(G))$.

Proof. Let $Z=Z(G)$ be the center of $G$. Let $\kappa=ccov(G/Z)$ and $(C_\alpha)_{\alpha\in\kappa}$ be a cover of $G/Z$ by cyclic subgroups. For every $\alpha\in\kappa$ choose an element $c_\alpha\in G$ such that the coset $c_\alpha Z$ is a generator of the cyclic subgroup $C_\alpha$. Observe that for every $\alpha\in\kappa$ the set $U_\alpha=\bigcup_{n\in\mathbb Z}c_\alpha^nZ$ is unfree and $\mathcal U=\{U_\alpha\}_{\alpha\in\kappa}$ is a cover of $G$ by unfree sets withessing that $$ucov(G)\le|\mathcal U|\le\kappa=|ccov(G/Z)|.\quad$$

Corollary. If a finite noncommutative group $G$ is almost squared, then $|Z(G)|\le \sqrt{|G|+1}-1$.

Proof. Let $A$ be an almost square root of $G$. By Theorem 1 and Lemma 1, we have $$|A|\le 1+ucov(G)\le 1+cc(G/Z(G))\le 1+|G/Z(G)|-1=|G|/|Z(G)|$$ and hence $$|Z(G)|\le \frac{|G|}{|A|}=\frac{|A|^2-1}{|A|}=|A|-\frac1{|A|}$$ and finally $$|Z(G)|\le -1+|A|=-1+\sqrt{|G|+1}.$$

Proposition 120. If a non-commutative group $G$ of order 120 is almost squared, then $|Z(G)|\le 6$.

Proof. Let $Z=Z(G)$ be the center of the group $G$ and $A$ be an almost square root of $G$. By the preceding Corollary, $|Z|\le 10$. Assuming that $|Z|=10$, we conclude that the group $G/Z$ has order 12. Looking at the classification of groups of order 12, we can observe that $G/Z$ contains a cyclic subgroup of order $\ge 4$. Then $ccov(G/Z)\le 1+(12-4)=9$ and hence $11=|A|\le 1+ucov(G)\le 1+ccov(G/Z)\le 10$, which is not true. This contradiction shows that $|Z|<10$ and hence $|Z|\le 8$.

Assuming that $|Z|=8$, we conclude that the group $G/Z$ has order $15$ and hence is cyclic, which implies $11=|A|\le 1+ucov(G)\le 1+ccov(G/Z)=2$ and this is a contradiction. So, $|Z|<8$ and hence $|Z|\le 6$ (as $|Z|$ divides $|G|=120$).

Proposition 168. If a non-commutative group $G$ of order 168 is almost squared, then $|Z(G)|\le 7$.

Proof. Let $Z=Z(G)$ be the center of the group $G$ and $A$ be an almost square root of $G$. By Corollary, $|Z|\le 12$. Assuming that $|Z|=12$, we conclude that the group $G/Z$ has order 14 and hence is cyclic. By Theorem 1, $$13=|A|\le 1+ucov(G)\le 1+ccov(G/Z)=2,$$which is a contradiction showing that $|Z|\ne 12$. Then $|Z|<12$ and hence $|Z|\le 8$ (11,10 and 9 do not divide $168=|G|$). If $|Z|=8$, then the group $G/Z$ has cardinality $168/8=21$ and hence is cyclic. Then $$13=|A|\le 1+ucov(G)\le 1+ccov(G/Z)=2,$$which is a contradiction showing that $|Z|\ne 8$. Therefore, $|Z|\le 7$.

Answer 3

Here is another example of a group with an almost square root. In a simple group $G$ of order 168 there exists a subset of 13 elements whose square is equal to $G$. We found several such subsets. Here is one of them. Let $G=PSL(3,2)=SL(3,2)={\rm gr\,}((4,6)(5,7),(1,2,4)(3,6,5))$. Then

$ A=\{ (1,2)(4,7),\ (1,4)(3,6),\ (1,2)(4,5,7,6),\ (1,2,3)(4,5,7),\ (1,2,3)(4,7,6),\\ (1,2,4,5,7,3,6),\ (1,2,4,7)(3,6),\ (1,2,6)(3,4,7),\ (1,2,6,3,4,5,7),\\ (1,5,2)(3,4,7),\ (1,5,2,3,6,4,7),\ (1,5,2,4,7,6,3),\ (1,5,2,6)(4,7) \} $

is an almost square root of $G$. We can use the following GAP commands to check this fact:

g:=PSL(3,2);

qr:=[ (1,2)(4,7), (1,4)(3,6), (1,2)(4,5,7,6), (1,2,3)(4,5,7), (1,2,3)(4,7,6), (1,2,4,5,7,3,6), (1,2,4,7)(3,6), (1,2,6)(3,4,7), (1,2,6,3,4,5,7), (1,5,2)(3,4,7), (1,5,2,3,6,4,7), (1,5,2,4,7,6,3), (1,5,2,6)(4,7)];;

prod:=[];; for a in qr do for b in qr do Add(prod,a*b);od;od;

Size(AsSet(prod));

Briefly, our recursive algorithm uses the function $F(A)$:

1. We start the computation from a set $A$ consisting of two elements of order 2.

2. Suppose we have already computed a set $A$ such that $|A^2|\geq|A|^2-1$.

3. We compute the set of candidates $C$ by the rule

$C=G\setminus(A^{-1}A^2\cup A^2A^{-1}\cup N_G(A)\cup S(A))$,

where $N_G(A)=\{x\in G\,\mid\,x^{-1}Ax\cap A\neq\emptyset \}$ and $S(A)=\{x\in G\,\mid\,x^2\in A^2\}$.

4. After that, for each $x\in C$ we compute $A'=\{A,x\}$ and run $F(A')$.

Note that, there are actually exactly three (up to conjugacy) pairs of elements of order 2 in the group $G$: $(12)(47),\ (15)(26)$; $(12)(47),\ (14)(36)$; $(12)(47),\ (13)(46)$.

Answer 4

Simple group of order 360

Another good example of an almost square simple group is the alternating group $A_6$ of order $360=19^2-1$. In order to check that the set $A$ below is an almost square root of $A_6$, you can use the following GAP commands:

g:=AlternatingGroup(6);;
A:=[(1,2)(3,4),(1,3)(5,6),(1,2,3,4)(5,6),(1,2,3),(1,3,4),
(1,2)(3,4,5,6),(1,2)(3,4,6,5),(1,2,3,4,5),(1,2,3,4,6),
(1,2,5)(3,4,6),(1,2,5,3,4),(1,2,5,6)(3,4),(1,2,5,6,3),
(1,2,6)(3,4,5),(1,2,6,3,4),(1,2,6,5)(3,4),(1,2,6,5,3),
(1,3,4,5,6),(1,3,4,6,5)];;
Size(A);
prod:=[];;
for a in A do for b in A do Add(prod,a*b);od;od;    
Size(AsSet(prod));

Note that the subset $$ V=\{(1,2)(3,4),(1,3)(5,6),(1,2,3,4)(5,6),(1,2,3),(1,3,4)\}\subset A $$ is an almost square root of a group isomorphic to the symmetric group $S_4$. The set $V$ is an initial set from which our algorithm starts. We next consider the following three 5-element subsets of $A_6$: \begin{eqnarray*} V_1&=&\{(1,2)(3,4),(1,3)(5,6),(1,2,3,4)(5,6),(1,2,3),(1,3,4)\},\\ V_2&=&\{(1,2)(3,4),(1,3)(5,6),(1,2,3,4)(5,6),(1,3,2),(2,3,4)\},\\ V_3&=&\{(1,2)(3,4),(1,3)(5,6),(1,2,3,4)(5,6),(1,4,2),(2,4,3)\}. \end{eqnarray*} Each of these subsets has the following three properties: (1) $V_i$ is an almost square root of a group isomorphic to $S_4$; (2) $V_i$ has a subset with three elements which is an almost square root of a group isomorphic to the dihedral group $D_4$ of order 8; (3) $V_i$ contains two elements of order 2.

It turns out that any 5-subset of $A_6$ with these three properties is conjugate to one of $V_i$ or $V_i^{-1}$ under the group Aut$(A_6)$. Each set $V_i$ is complemented to an almost square root of $A_6$.

Groups of odd order

It seems to me that the answer to Problem 3' is negative.

Proposition. Any group of order $675$ is not an almost squared group.

Proof. This is a straightforward consequence of Lemmas 1 and 2.

Lemma 1. Let $G$ be a group of odd order and $|G|>3$. If $G$ has an almost square root, then $Z(G)=\{e\}$.

Proof. Let $A$ be an almost square root of $G$. Then there exists exactly one element $x\in G$ which can be represented as the product $x=ab$, $a,\,b\in A$ in exactly two ways. We will call this element a singular element of group $G$ with respect to the almost square root $A$. All the other elements of group $G$ are represented uniquely as the product of two elements of $A$.

If $Z(G)\neq\{e\}$ and $z\in Z(G)$, $z\neq e$, then there are two possible cases, either $z=a^2$ where $a\in A$ or $z=ab$, $a\neq b$ and $a,b\in A$.

In the first case we have $z=a^2$. Since the order of $G$ is odd, it follows that $a\in Z(G)$ and $ab=ba$ for each $b\in A$. Hence $|A|=2$ and $|G|=3$.

In the second case we have $z=ab=ba$ and $z$ is singular element. Since there is exactly one singular element, the equality $z^2=a'b'$ with $a',b' \in A$, $a'\neq b'$, is impossible. Then $z^2=a'^2$, $a'\in A$, and we are back to the first case. This is a contradiction and the lemma is proved.

Lemma 2. Let $G$ be a group of order $675=3^3\cdot5^2$. Then $Z(G)\neq\{e\}$.

Proof. Using the small groups library of GAP we can prove of Lemma 2 by the following GAP commands:

a:=AllSmallGroups(675);;
Filtered(a,x->Size(Center(x))=1);

However, the same can easily be proved without GAP.

Answer 5

Here are additional considerations for groups with an almost square root. This is a development of the idea of Taras Banakh.

Let $G$ be a finite group. The following graph $\Gamma=\Gamma(G)$ associated with group $G$ is useful. The graph $\Gamma$ has as vertex set the elements of the group $G$, two vertices $x$, $y$, $x\neq y$, being joined if $xy = yx$ or $x^2=y^2$. A subset $X$ of the vertices of $\Gamma$ is called an independent set if the induced subgraph on $X$ has no edges. The independence number of $\Gamma$ is the maximum size of an independent set of vertices and is denoted by $\alpha(\Gamma)$.

Lemma. If $A$ is an almost square root of $G$, then $|A|\leq\alpha(\Gamma(G))+1$.

Here is an example of application of the lemma. There are a total of 190 non-Abelian groups of order $224=15^2-1$. For each of them the graph $\Gamma$ and its independence number are calculated using the GAP and GRAPE packages. The maximum independence number for these groups is $10<16$.

RequirePackage( "grape" );
A:=[];;
all:=AllSmallGroups(224,IsAbelian,false);;
for g in all do
    v:=SortedList(g);;
    A:=NullMat(Size(v),Size(v));;
    for x in v do for y in v do if x*y=y*x and x<>y then A[Position(v,x)][Position(v,y)]:=1;fi;od;od;
    for x in v do for y in v do if x^2=y^2 and x<>y then A[Position(v,x)][Position(v,y)]:=1;fi;od;od;
    gamma:=Graph(Group(()), [1..Size(v)], OnPoints,function(x,y) return A[x][y]=1; end, true );;
    root:=IndependentSet( gamma );;
    Print(Position(all,g),", ", Size(root),", ", StructureDescription(g),"\n");
od;

Answer 6

The following theorem gives a partial answer to Problem 2 on the structure of almost squared groups. Let us recall that the Abelianization of a group $G$ is the quotient group $G/[G,G]$ of $G$ by its commutator subgroup.

Theorem. Let $G$ be an almost squared group and $R$ be an almost square root of $G$. Let $H=[G,G]$ be the commutator subgroup of $G$ and $A=G/H$ be the Abelianization of $G$. Then there exists $\epsilon\in\{-1,1\}$ such that:

1. $|A|$ divides $|R|+\epsilon$.

2. The set $L=\big\{gH\in A:|R\cap gH|=\frac{|R|+\epsilon}{|A|}\big\}$ has cardinality $|L|=|A|-1$.

3. For the unique element $gH$ of the singleton $A\setminus L$ we have $|R\cap gH|=\frac{|R|+\epsilon}{|A|}-\epsilon$.

Proof. For a subset $S\subseteq G$ let $\sum S$ be the element $\sum_{x\in S}x$ in the group algebra $\mathbb C G$. Taking into account that $R$ is an almost square root of $G$, we conclude that $(\sum R)^2=\sum G+s$ for some element $t\in G$ (having two representations as products of elements of the set $R$).

In the group algebra $\mathbb C A$ of the Abelianization $A=G/H$, consider the element $y=\sum_{gH\in A}|R\cap gH|\cdot gH$. Since $R$ is an almost square root of $G$, $(\sum R)^2=\sum G+t$ for some elememt $t\in G$ (that has a double representation as a product of two elements in $R$). The equality $(\sum R)^2=\sum G+t$ implies $y*y=|H|\cdot\sum A+\tau$ where $\tau=tH\in A$. Now consider the elements $z=y-\frac{|R|\pm1}{|A|}\sum A$ of $\mathbb C A$ and observe that $$ \begin{aligned} z*z&=(y-\tfrac{|R|\pm1}{|A|}\sum A)*(y-\tfrac{|R|\pm1}{|A|}\sum A)=\\ &=y*y-\tfrac{|R|\pm1}{|A|}\sum_{b\in A}y*b-\tfrac{|R|\pm1}{|A|}\sum_{b\in A}b*y+\tfrac{(|R|\pm1)^2}{|A|^2}\sum_{a,b\in A}ab=\\ &=|H|\cdot\sum A+\tau-\tfrac{|R|\pm1}{|A|}\sum_{a,b\in A}|R\cap a|\cdot (ab+ba)+\tfrac{(|R|\pm1)^2}{|A|}\sum A=\\ &=\tfrac{|H|\cdot|A|+(|R|\pm1)^2}{|A|}\cdot\sum A+\tau-2\tfrac{|R|\pm1}{|A|}|R|\sum A=\\ &=\tfrac{|G|+|R|^2\pm2|R|+1-2(|R|\pm1)|R|}{|A|}\sum A+\tau=\\ &=\tfrac{|R|^2-1-|R|^2+1}{|A|}\sum A+\tau=\tau. \end{aligned} $$

Let $\hat z$ be the Fourrier transformation of $z$. It is the function assigning to every character $\chi:A\to\mathbb C$ the complex number $$\hat z(\chi)=\sum_{a\in A}z(a)\chi(a)$$where $z=\sum_{a\in A}z(a)\cdot a$. Since the function $\hat \tau$ has absolute value $1$ at each character, $\hat z\cdot \hat z=\widehat{z{*}z}=\hat \tau$, the function $\hat z:\hat A\to\mathbb C$ also has its values in the unit circle and hence $\hat z$ has norm $1$ in the Hilbert space $\mathbb C\hat A$ endowed with the inner product $$\langle f,g\rangle=\frac1{|A|}\sum_{a\in \hat A}f(a)\cdot\overline{g(a)}.$$ Endow the group algebra $\mathbb C A$ with the inner product $$\langle f,g\rangle=\sum_{a\in A}f(a)\cdot\overline{g(a)}.$$ Since the Fourrier transformation is an isometry isomorphism of the Hilbert spaces $\mathbb C A$ and $\mathbb C\hat A$, we obtain that $$ (\star)\quad\|z\|^2=\sum_{a\in A}\big||R\cap a|-\tfrac{|R|\pm 1}{|A|}\big|^2=\|\hat z\|=1. $$ Since $|A|$ divides $|G|=|R|^2-1$, the number $\frac{|R|}{|A|}$ is not integer. Let $\varepsilon>0$ be the smallest natural number such that $\frac{|R|+\varepsilon}{|A|}$ is integer.

The equation ($\star$) implies that for every $a\in A=G/H$ the number $|R\cap a|$ is equal either to $\frac{|R|+\varepsilon}{|A|}$ or to $\frac{|R|+\varepsilon}{|A|}-1$. Let $$P=\{a\in A:|R\cap a|=\tfrac{|R|+\varepsilon}{|A|}\}\quad\mbox{and}\quad Q=\{a\in A:|R\cap a|=\tfrac{|R|+\varepsilon}{|A|}-1\}.$$ Then the equation ($\star$)reduces to the equation $$(\star\star)\quad \begin{aligned} |A|^2&=|P|\cdot(\varepsilon\mp1)^2+|Q|\cdot(\varepsilon\mp1-|A|)^2=\\ &=|P|\cdot(\varepsilon\mp 1)^2+|Q|\cdot(\varepsilon\mp 1)^2-2|Q|\cdot|A|\cdot (\varepsilon\mp1)+|Q|\cdot|A|^2=\\ &=|A|\cdot(\varepsilon\mp 1)^2-2|Q|\cdot|A|\cdot(\varepsilon\mp1)+|Q|\cdot|A|^2 \end{aligned} $$ as $|P|+|Q|=|A|$.

Subtracting the equations ($\star$) for different signs after $\varepsilon$, we obtain obtain that $\varepsilon=|Q|$. After substitution of $\varepsilon=|Q|$ into the equation ($(\star\star$) we obtain that $\varepsilon=|Q|$ is equal to $1$ or $|A|-1$.

If $\varepsilon=|Q|=1$, then $|A|$ divides $|R|+\varepsilon=|R|+1$. If $\varepsilon=|Q|=|A|-1$, then $|A|$ divides $|R|+\varepsilon=|R|+|A|-1$ and hence divides $|R|-1$. Put $\epsilon=1$ if $\varepsilon=1$ and $\epsilon=-1$ if $\varepsilon=|A|-1$. In both cases we obtain that $|A|$ divides $|R|+\epsilon$.

Observe that $$|H|=\frac{|G|}{|A|}=\frac{|R|^2-1}{|A|}=\frac{|R|+\epsilon}{|A|}(|R|-\epsilon),$$ which means that $|H|$ is divisible by $|R|-\epsilon$.

If $|Q|=1$, then $\epsilon=1$, the set $$L=\{gH\in A:|R\cap gH|=\tfrac{|R|+\epsilon}{|A|}\}=|P|$$ has cardinality $|P|=|A|-|Q|=1$, and $$\{gH:|R\cap gH|=\tfrac{|R|+\epsilon}{|A|}-\epsilon\}$$ is a singleton.

If $|Q|=|A|-1$, then $\epsilon=-1$, the set $$\{gH:|R\cap gH|=\tfrac{|R|+|A|-1}{|A|}-1\}= \{gH:|R\cap gH|=\tfrac{|R|+\epsilon}{|A|}\}=L$$ has cardinality $|A|-1$.

On the other hand, the set $$P=\{gH\in A:|R\cap gH|=\tfrac{|R|+|A|-1}{|A|}\}=\{gH\in A:|R\cap gH|=\tfrac{|R|+\epsilon}{|A|}-\epsilon\}$$ is a singleton.