Let $G$ be a compact connected simple Lie group, $T$ a maximal torus, $N(T)$ the normalizer of $T$, and $W=N(T)/T$ the Weyl group. It is wellknown that $H^*(G/T,\mathbb{Q})$ is the regular representation of $W$ induced by the $W$action on $G/T$, and hence $H^*(G/N(T), \mathbb{Q})\cong H^*(G/T, \mathbb{Q})^W\cong\mathbb{Q}$. I would like to know whether the integral cohomology $H^*(G/N(T), \mathbb{Z})$ has been known before. I am particularly interested in the case $G=SU(n)$.

3$\begingroup$ I advocate using $N(T)$ to denote this group, since $N$ often gets used for the unipotent group. (Admittedly noone would ask about the cohomology of $G/N$ with that meaning, since that $N$ is contractible.) $\endgroup$ – Allen Knutson May 18 '16 at 13:14

$\begingroup$ I agree. I changed the notation for the normalizer accordingly. $\endgroup$ – Alex Fok May 18 '16 at 16:11

$\begingroup$ If I am not mistaken this is worked out by Manin already? $\endgroup$ – Bombyx mori May 18 '16 at 17:07

3$\begingroup$ Here is a sign that this may be hard. We have a spectral sequence $H^q(W, H^p(\mathcal{F})) \to H^{p+q}(\mathcal{F}/W)$ where the left hand side is group cohomology. Take a look at the $p=0$ row for $SU(n)$: tables of $H^q(S_n, \mathbb{Z})$ can be found at groupprops.subwiki.org/wiki/… . They aren't simple... $\endgroup$ – David E Speyer May 24 '16 at 9:34

1$\begingroup$ Something neat happens when $p=2$. We have a short exact sequence $0 \to \mathbb{Z} \to \mathbb{Z}^n \to H^2(\mathcal{F}) \to 0$, with the obvious actions. We can describe $\mathbb{Z}^n$ as $\mathrm{Ind}_{S_{n1}}^{S_n} \mathbb{Z}$, so $H^q(S_n, \mathbb{Z}^n) \cong H^q(S_{n1}, \mathbb{Z})$. Thus, the terms in $p=2$ measure the failure of $H^q(S_{n1}, \mathbb{Z}) \to H^q(S_n, \mathbb{Z})$ to be be an isomorphism  in other words, the lack of stabilization. If we fix $q$ and send $n \to \infty$, they are eventually $0$. $\endgroup$ – David E Speyer May 24 '16 at 9:44
There are people on MO more knowledgeable about this topic than I, but since it has been a few days, I will say a few words.
In this case there is a "third isomorphism" of spaces:
Proposition $G/N(T)\cong \mathcal{F}/W$
Proof The flag variety $\mathcal{F}\cong G/T\cong G_{\mathbb{C}}/B$ and $W=N(T)/T$ is the Weyl group. Define $\varphi:G/T\to G/N(T)$ by $gT\mapsto gN(T)$. This is clearly a welldefined continuous surjection. The group $W$ acts on the right of $G/T$ (and $G/N(T)$) by $gT\cdot nT=gTnT=gnT$ and $\varphi$ is $W$equivariant. So $\varphi$ descends to a map $\varphi:\mathcal{F}/W\to G/N(T)$ which is easily seen to be injective, and hence a homeomorphism (since $\mathcal{F}/W$ is compact and $G/N(T)$ is Hausdorff). $\Box$
As shown here there is an inclusion $H^k(\mathcal{F}/W;\mathbb{Z})\hookrightarrow H^k(\mathcal{F};\mathbb{Z})^W$ modulo torsion. Since you know $H^*(\mathcal{F}/W;\mathbb{Q})\cong H^*(\mathcal{F};\mathbb{Q})^W$ (by a result in Bredon's book on Transformation Groups; page 120) and the latter is $\mathbb{Q}$ (by Borel's 1953 Annals paper, page 193) you know the free parts since $\mathrm{rk} \ H^k(\mathcal{F}/W;\mathbb{Z})=\mathrm{rk}\ H^k(\mathcal{F};\mathbb{Z})^W.$
That leaves torsion. I spoke with an expert in Shubert varieties who told me that (a) they did not know the answer or a reference to this, and (b) they expected complicated torsion.
So I decided to do an example. First note that $W$ acts freely on $\mathcal{F}$ since if $gwT=gT$, then $w\in T$. Thus, $\mathcal{F}/W$ is a smooth manifold.
Example: $\mathrm{SU}(2)/T\cong S^2$ with $W=\mathbb{Z}/2\mathbb{Z}$ acting freely, so the quotient must be $\mathbb{RP}^2$. That gives torsion in $H^2(\mathcal{F}/W;\mathbb{Z})$ at the first opportunity.
More generally, when $G$ is simply connected $G/T$ is simply connected (by path lifting), and so $H_1(\mathcal{F}/W;\mathbb{Z})\cong W/[W,W]$. This can be used with Poincaré duality (or a nonorientable version) to garner some further information (in particular, lots of torsion as $W$ gets larger).

1$\begingroup$ This is a great answer. I wish I could have written this answer! Upvoted. $\endgroup$ – Bombyx mori May 24 '16 at 0:26

2$\begingroup$ $W/[W,W]$ is always of the form $(\mathbb{Z}/2)^s$. To find $s$, take the Dynkin diagram and declare two vertices to be equivalent if they are joined by an $A_2$ edge. Then $s$ is the number of equivalence classes generated by this. $\endgroup$ – David E Speyer May 24 '16 at 8:55

1$\begingroup$ Also, as the original post says, $H^{\ast}(\mathcal{F}, \mathbb{Q})$ is the reqular representation of $W$, so the torsion free part is just one dimensional in $H^0$. $\endgroup$ – David E Speyer May 24 '16 at 9:06

1$\begingroup$ @SeanLawton and DavidSpeyer Thank you very much for your input! I especially like your observation about $H_1(\mathcal{F}/W, \mathbb{Z})$. This is a good place to start. $\endgroup$ – Alex Fok May 24 '16 at 12:05

3$\begingroup$ @AlexFok No worries! Making mistakes is how we learn, especially in mathematics :) $\endgroup$ – Sean Lawton May 24 '16 at 14:33
Here are some comments about how this calculation can be organised.
First, let $T(n)$ be the standard maximal torus in $U(n)$, and let $N(n)$ be its normaliser, so $W(n)=N(n)/T(n)$ is the symmetric group. Put $F(n)=U(n)/T(n)$ and $E(n)=F(n)/W(n)=U(n)/N(n)$, so we are interested in the (co)homology of $E(n)$. We know that $H^*(E(n);\mathbb{Q})=\mathbb{Q}$. I think it will be best to study $H_*(E(n);\mathbb{Z}/p)$ (where $p$ is prime) and recover information about $H^*(E(n);\mathbb{Z})$ at the end, using the universal coefficient theorems. From now on I'll write $H_*(X)$ for $H_*(X;\mathbb{Z}/p)$. It will also be best to study all the groups $H_*(E(n))$ together as a bigraded object $H_*(E(*))$.
Note that there are sum and tensor product operations $\oplus\colon U(n)\times U(m)\to U(n+m)$ and $\otimes\colon U(n)\times U(m)\to U(nm)$. These are group homomorphisms, so they induce maps $BU(n)\times BU(m)\to BU(n+m)$ and $BU(n)\times BU(m)\to BU(nm)$. They preserve monomial matrices, so they give maps $BN(n)\times BN(m)\to BN(n+m)$ and $BN(n)\times BN(m)\to BN(nm)$. We can regard $E(n)$ as the fibre of the map $BN(n)\to BU(n)$, so we also get maps $E(n)\times E(m)\to E(n+m)$ and $E(n)\times E(m)\to E(nm)$. This will give two different products on the object $H_*(E(*))$, which will make it into a Hopf ring. Also, using block matrices we get a standard inclusion $\Sigma_n\wr U(m)^n\to U(nm)$. This gives an action of an $E_\infty$ operad on $BU(*)$ and $BN(*)$ and $E(*)$, and so gives DyerLashof operations on the corresponding homology groups.
Now $BN(*)$ is essentially the free $E_\infty$ space generated by $BN(1)=\mathbb{C}P^\infty$, which implies that $H_*(BN(*))$ is the free commutative algebra with DyerLashof operations generated by $H_*(\mathbb{C}P^\infty)$. On the other hand, $H_*(BU(*))$ is just the free commutative algebra generated by $H_*(\mathbb{C}P^\infty)$. This implies that the map $H_*(BN(*))\to H_*(BU(*))$ is surjective (which can also be seen using the BeckerGottlieb transfer map). Note also that the DyerLashof operations in $H_*(BU(*))$ were worked out by Priddy. It seems likely that $H_*(E(*))$ is just the kernel (in the Hopf algebra sense) of the map $H_*(BN(*))\to H_*(BU(*))$. Some care is needed here because the relevant Hopf algebras do not have antipodes; they correspond to commutative monoids (rather than groups) in the category of coalgebras. Anyway, the message is that we expect that $H_*(E(*))$ will have structure closely related to that of the DyerLashof algebra, and of similar size and complexity.
So far we have not used the tensor product maps $U(n)\times U(m)\to U(nm)$, or equivalently, the second product in the Hopf ring structure. It is a theorem of Paul Turner that when $p=2$, the object $H_*(BW(*))$ has a simple presentation as a Hopf ring, which is in some ways more tractable than the presentation using DyerLashof operations. I expect that there is a similar story for $H_*(BN(*))$, although I do not know whether that has ever been written down. It is also straightforward to write down the Hopf ring structure on $H_*(BU(*))$, and this may well give a nice description of $H_*(E(*))$. I am not sure whether a similar approach would work for $p>2$.