Positivity of the alternating sum of indices for boolean interval of finite groups

Question

Let $G$ be a finite group and $H$ a subgroup such that the interval $[H,G]$ is a boolean lattice.

Let $L_1, \dots , L_n$ be the maximal subgroups of $G$ containing $H$.
Let the alternative sum of the interval $[H,G]$ defined as follows:
$$\chi([H,G]):= \sum_{r=0}^n (-1)^{r} \sum_{ \ i_1 < i_2 < \cdots < i_r } [L_{i_1} \wedge \cdots \wedge L_{i_r}: H] $$ Notation: $L_{i_1} \wedge \cdots \wedge L_{i_r} = G$ for $r=0$.

Theorem: $\chi([H,G]) > 0$.
Proof: Observe that
$$\chi([H,G]) = \frac{\vert G \vert - \vert \bigcup_i L_i \vert}{\vert H \vert} $$ but a boolean lattice is distributive so by a result of Oystein Ore (see here) $\exists g \in G$ with $\langle H,g \rangle = G$, which precisely means that $g \not \in L_i \ \forall i$, and so $\chi([H,G])> 0$ $\square$

Let $K_1, \dots , K_n$ be the minimal overgroups of $H$.
Let the dual alternative sum of the interval $[H,G]$ defined as follows:
$$\hat{\chi}([H,G]):= \sum_{r=0}^n (-1)^{r} \sum_{ \ i_1 < i_2 < \cdots < i_r } [G: K_{i_1} \vee \cdots \vee K_{i_r}] $$ Notation: $K_{i_1} \vee \cdots \vee K_{i_r} = H$ for $r=0$.

Question: Is $\hat{\chi}([H,G]) > 0$ ?
Remark: after GAP checking, it is true for $[G:H]<32$ (recall that $[H,G]$ is assumed boolean).

Answer 1

UPDATE: The original poster of the question, together with Mamta Balodi, have shown that the labeling I suggest below is an EL-labeling if and only if group (product) complements coincide with lattice complements on the given Boolean interval in the subgroup lattice. The latter condition does not always hold; in the same paper they present examples where it does not. The Cohen-Macaulay and/or shellability questions still seem to be open in general.

Unless I'm mistaken, this is (up to sign) the Möbius number of the poset $C(H,G)$ formed by all cosets of all groups on $[H,G]$, together with an artificial $\hat{0}$ element. This is because the Möbius number of a Boolean interval is $\pm 1$. Thus, your sum is really

$$ (-1)^n \cdot \sum_{H \leq K \leq G} \mu(K,G) \cdot [G:K].$$ The Möbius number observation I made now follows from definition.

I believe that $C(H,G)$ is Cohen-Macaulay. Since the Möbius number of a Cohen-Macaulay poset of height $n$ is positive or negative depending on whether $n$ is even or odd, this would give you the desired positivity.

There are lots of ways to prove a poset to be Cohen-Macaulay. One way would be to construct a dual EL-labeling. I think that you can do this as follows, but have not written down a proof. (Edit: the following works some of the time, but not all of the time; see UPDATE above.) Label the bottom edges $\hat{0} \lessdot Hh$ with 0. Let $M_i$ be the join of all $K_j$ except for $K_i$, so that $M_i$ is maximal in $[H,G]$ for any $i$. If $Y = X \vee K_i$, then label $Xh \lessdot Yh$ with $-i$ if $Xh = Yh \wedge M_i$, and label $Xh \lessdot Yh$ with $+i$ otherwise.

Now verify (assuming it is actually true) that every interval has a unique increasing chain that is lexicographically first, as required for an EL-labeling.

The labeling that I suggest here is extremely similar to that constructed in Section 4.2 of my paper "Cubical convex ear decompositions". The observation that the quantity you're interested in is a Möbius number is attributed to Bouc, but is in a paper of Ken Brown, "The coset poset and probabilistic zeta function of a finite group". The latter is rather well-written, and also discusses some other ways of computing the Möbius function of the full coset lattice.