I will try to post a full answer in the next few days. In the meanwhile you may be interested in the proof for the case $k=2$ given in eprint.iacr.org/2005/318 (Theorem 3).

@kodlu: it was only a short comment, not a full answer.

I had given an answer (roughly 2 years ago) in your blog about this, did you see that?

No, I wanted to say: for any (admitted) $f$ we have (by Jensen's inequality) $\mathbb{E}\big[\big(\mathbb{E}(f(Y)|X,Z)\big)^2|X\big]\geq \big[\mathbb{E}(f(Y)|X,Z)|X\big]^2= \big[\mathbb{E}(f(Y)|X)\big]^2$ (a.s.), and thus $\mathbb{E}\big[\big(\mathbb{E}(f(Y)|X,Z)\big)^2]\geq \mathbb{E} \big[\big(\mathbb{E}(f(Y)|X)\big)^2\big]$. The rest is obvious.

I think this questions should be moved to MSE. (Hint: in your final formulation, just condition the integrand on the lhs with resp. to $X$ , and use Jensen's inequality for conditional expectations, before taking expectations.)

After writing your matrix in the form $(A_{i,j})_{1\leq i,j \leq p-1}$ with elements $A_{i,j}=(1+i-j)^{p-2}\in \mathbb{F}_p$ it is not difficult to show that the determinant is indeed $1\;\bmod p$ if $p\geq 3$ is prime. Is that still of interest?

A cached copy of Bizley's paper is here oeis.org/A060941

The result was proved by Simon Webb in his PhD thesis discovery.ucl.ac.uk/id/eprint/10102127/1/out.pdf (If you were aware of that you should have stated it.)

For the exponential case take a look at this paper projecteuclid.org/download/pdf_1/euclid.aoms/1177706269 . The distribution of the last exit time for a Brownian motion with drift was also described here mathoverflow.net/questions/222705/…

In queueing theory this distribution is called the Borel distribution with arrival rate $a$. It is known that it decribes the the number of customers in a busy period of an $M/D/1$ -queue with arrival rate $a$ and deterministic service time 1 - this is essentially equivalent to the statement about $T_a$. I' ll have to look for a moe direct reference.