I have the following conjecture involving a possible new formula for the class number of the quadratic field $\mathbb Q(\sqrt{(-1)^{(p-1)/2}p})$ with $p$ an odd prime.
Conjecture. Let $p$ be an odd prime and let $p^*=(-1)^{(p-1)/2}p$. Then the class number $h(p^*)$ of the quadratic field $\mathbb Q(\sqrt{p^*})$ coincides with the number
$$D(p):=\frac{(\frac{-2}p)}{2^{(p-3)/2}p^{(p-5)/4}}\det\left[\cot\pi\frac{jk}p\right]_{1\le j,k\le (p-1)/2},$$
where $(\frac{\cdot}p)$ is the Legendre symbol.
This is Conjecture 5.1 in my preprint arXiv:1901.04837. I have checked it for all odd primes $p<29$. Note that $h(p^*)=1$ for each odd prime $p<23$, and $h(-23)=3$.
Here I invite some of you to check this conjecture further. My computer cannot check it even for $p=29$.
Edit: F. Brunault computered $D(p)$ for $p=29,31,37,41,43,47$ and noted that the conjecture is false. However, I believe that $h(p^*)\mid D(p)$ for any odd prime $p$.
Now I explain why $D(p)\in\mathbb Q$ by Galois theory. The Galois group $\text{Gal}(\mathbb Q(e^{2\pi i/p})/\mathbb Q)$ consisits of those authormorphisms $\sigma_a$ $(1\le a\le p-1)$ with $\sigma_a(e^{2\pi i/p})=e^{2\pi ia/p}$. By Gauss' Lemma, $$\left(\frac ap\right)=(-1)^{|\{1\le j\le(p-1)/2:\ \{aj/p\}>1/2\}|}.$$ For $j=1,\ldots,(p-1)/2$ let $\pi_a(j)$ be the unique $r\in\{1,\ldots,(p-1)/2\}$ with $aj\equiv \pm r\pmod p$. Then, for $D=\det[\cot\pi\frac{jk}p]_{1\le j,k\le(p-1)/2}$, using $p$th roots of unity we get \begin{align}\sigma_a\left(\frac D{i^{(p-1)/2}}\right)=&\frac1{i^{(p-1)/2}}\det\left[\cot\pi\frac{ajk}p\right]_{1\le j,k\le(p-1)/2} \\=&\frac{(\frac ap)}{i^{(p-1)/2}}\det\left[\cot\pi\frac{\pi_a(j)k}p\right]_{1\le j,k\le(p-1)/2} \\=&\left(\frac ap\right)\left(\frac ap\right)^{(p+1)/2}\frac{D}{i^{(p-1)/2}}=\left(\frac ap\right)^{(p-1)/2}\frac{D}{i^{(p-1)/2}}\end{align} since $\text{sign}(\pi_a)=(\frac ap)^{(p+1)/2}$ as pointed by Pan in arXiv:0601026. Thus, if $p\equiv1\pmod4$ then $\sigma_a(D)=D$ for all $a=1,\ldots,p-1$ and hence $D\in\mathbb Q$. When $p\equiv3\pmod4$, we have $$\sigma_a\left(\frac D{\sqrt{p}}\right)=\left(\frac ap\right)\frac D{i^{(p-1)/2}}\cdot\frac1{\sigma_a(\sqrt{p^*})}.$$ Using Gauss' sums we see that $$\sigma_a(\sqrt{p^*})=\sum_{x=0}^{p-1}e^{2\pi iax^2/p}=\left(\frac ap\right)\sqrt{p^*}.$$ Therefore $\sigma_a(D/\sqrt p)=D/\sqrt p$ for all $a=1,\ldots,p-1$, and hence $D/\sqrt p\in\mathbb Q$.