I'm not really trying to convince anyone of anything - I've just seen a pattern and I'm trying to understand it. But you're right, with irregularly spaced $t_j$ the proportionality breaks down. I'll edit to reflect your comment.

I get a similar tridiagonal structure if I invert the gram matrix of the covariance of an Ornstein Uhlenbeck process. The plot thickens...

It was a long time ago and not my field - I think someone mentioned a result along those lines in a lecture I saw. This might be the right reference: arxiv.org/abs/math/9901005