# Positivity of the alternating sum of indices for boolean interval of finite groups

Let $G$ be a finite group and $H$ a subgroup such that the interval $[H,G]$ is a boolean lattice.

Let $L_1, \dots , L_n$ be the maximal subgroups of $G$ containing $H$.
Let the alternative sum of the interval $[H,G]$ defined as follows:
$$\chi([H,G]):= \sum_{r=0}^n (-1)^{r} \sum_{ \ i_1 < i_2 < \cdots < i_r } [L_{i_1} \wedge \cdots \wedge L_{i_r}: H]$$ Notation: $L_{i_1} \wedge \cdots \wedge L_{i_r} = G$ for $r=0$.

Theorem: $\chi([H,G]) > 0$.
Proof: Observe that
$$\chi([H,G]) = \frac{\vert G \vert - \vert \bigcup_i L_i \vert}{\vert H \vert}$$ but a boolean lattice is distributive so by a result of Oystein Ore (see here) $\exists g \in G$ with $\langle H,g \rangle = G$, which precisely means that $g \not \in L_i \ \forall i$, and so $\chi([H,G])> 0$ $\square$

Let $K_1, \dots , K_n$ be the minimal overgroups of $H$.
Let the dual alternative sum of the interval $[H,G]$ defined as follows:
$$\hat{\chi}([H,G]):= \sum_{r=0}^n (-1)^{r} \sum_{ \ i_1 < i_2 < \cdots < i_r } [G: K_{i_1} \vee \cdots \vee K_{i_r}]$$ Notation: $K_{i_1} \vee \cdots \vee K_{i_r} = H$ for $r=0$.

Question: Is $\hat{\chi}([H,G]) > 0$ ?
Remark: after GAP checking, it is true for $[G:H]<32$ (recall that $[H,G]$ is assumed boolean).

• Can we expect $\hat{\chi}([H,G]) \ge 2^{n-1}$? – Sebastien Palcoux Apr 8 '16 at 11:58

## 1 Answer

UPDATE: The original poster of the question, together with Mamta Balodi, have shown that the labeling I suggest below is an EL-labeling if and only if group (product) complements coincide with lattice complements on the given Boolean interval in the subgroup lattice. The latter condition does not always hold; in the same paper they present examples where it does not. The Cohen-Macaulay and/or shellability questions still seem to be open in general.

Unless I'm mistaken, this is (up to sign) the Möbius number of the poset $C(H,G)$ formed by all cosets of all groups on $[H,G]$, together with an artificial $\hat{0}$ element. This is because the Möbius number of a Boolean interval is $\pm 1$. Thus, your sum is really

$$(-1)^n \cdot \sum_{H \leq K \leq G} \mu(K,G) \cdot [G:K].$$ The Möbius number observation I made now follows from definition.

I believe that $C(H,G)$ is Cohen-Macaulay. Since the Möbius number of a Cohen-Macaulay poset of height $n$ is positive or negative depending on whether $n$ is even or odd, this would give you the desired positivity.

There are lots of ways to prove a poset to be Cohen-Macaulay. One way would be to construct a dual EL-labeling. I think that you can do this as follows, but have not written down a proof. (Edit: the following works some of the time, but not all of the time; see UPDATE above.) Label the bottom edges $\hat{0} \lessdot Hh$ with 0. Let $M_i$ be the join of all $K_j$ except for $K_i$, so that $M_i$ is maximal in $[H,G]$ for any $i$. If $Y = X \vee K_i$, then label $Xh \lessdot Yh$ with $-i$ if $Xh = Yh \wedge M_i$, and label $Xh \lessdot Yh$ with $+i$ otherwise.

Now verify (assuming it is actually true) that every interval has a unique increasing chain that is lexicographically first, as required for an EL-labeling.

The labeling that I suggest here is extremely similar to that constructed in Section 4.2 of my paper "Cubical convex ear decompositions". The observation that the quantity you're interested in is a Möbius number is attributed to Bouc, but is in a paper of Ken Brown, "The coset poset and probabilistic zeta function of a finite group". The latter is rather well-written, and also discusses some other ways of computing the Möbius function of the full coset lattice.

• This answer looks very promising so I award the bounty. I will study it in detail as soon as possible. – Sebastien Palcoux Apr 2 '16 at 18:57
• You wrote << I believe that $C(H,G)$ is Cohen-Macaulay, which would give you the desired. There are lots of ways to prove this >>. What do you mean by this? Is it "Cohen-Macaulay" or "the desired"? And how do you get the expected strict positivity? – Sebastien Palcoux Apr 3 '16 at 11:44
• There are lots of ways to prove a complex to be Cohen-Macaulay. Actually, you only need for the order complex to have the homology of a bouquet of spheres in a fixed dimension with the right parity. I edited the post for clarity. – Russ Woodroofe Apr 4 '16 at 0:17
• You might look at (4.1.1) of "Poset topology: tools and applications", by Michelle Wachs. That survey paper also explains why this holds. It is possible for a Cohen-Macaulay poset to have 0 as its Möbius number, for example if it is contractible. That's maybe another reason to use the EL-labeling machinery -- that gives you an explicit calculation for the Möbius number as the number of decreasing chains. See Theorem 3.2.4 of the same survey paper. – Russ Woodroofe Apr 4 '16 at 17:25
• You should mention that the labeling appearing in your answer is now proved to be a dual EL-labeling iff the interval is group-complemented (see this paper, Theorem 4.23 p19). – Sebastien Palcoux May 28 '16 at 17:51