This is an attempt to clarify my comments; hopefully it will of use to the OP in determining how to use their formula. Like @JimHuphreys, I am not familiar with the particular version of the Weyl Character Formula (WCF) used by the OP (and in particular it seems a bit odd that it doesn't seem to involve the Weyl Group of $SO(2n)$ like the usual WCF does. If anyone else sees how the sinh/cosh take care of that, do enlighten me; perhaps it is related to the fact that we are working with $SO(2n)$ instead of its universal cover). Nevertheless, it seems most of your problem is stemming from an incorrect interpretation of the inputs as I shall try to outline below.

Anyhow, the quick background: $SO(2n)$ comes from the $D_n$ family of Lie groups; therefore it has rank $n$ and a highest weight of a representation is determined by a non-negative $n$-tuple of integers, say $(a_1,\ldots,a_n)$. The coordinates in this expression are the coordinates relative to a basis of fundamental weights for the group; these in turn are highest weights of the corresponding fundamental representations, so the weight can also be written as $\sum_{i=1}^n a_i\omega_i$ where the $\omega_i$ are these fundamental weights.

Now when I think of weights, I think of them in terms of logarithms of eigenvalues: when one applies the WCF one starts with a highest weight $\sum_{i=1}^n h_i\omega_i$ (where the $h_i$ *should* be the same as the ones in your formula) and applies each element of the Weyl group $W(G)$ to this weight according to a formula which looks a bit weird at first, but turns out to be fairly easy to program into a computer (at least when $W(G)$ is not too big). Each weight resulting from this action on the weight space is of the form $\sum_{i=1}^n b_i\omega_i$ where the $b_i$ are integral but no longer restricted to being all nonnegative. Each such weight then corresponds to a term of the form $\pm e^{b_1t_1+b_2t_2+\ldots+b_nt_n}$ which appears in the numerator of the WCF; the denominator of the formula has similar terms formed by applying the $W(G)$-action to the highest weight with all coordinates equal to 0.

To evaluate the character of the irreducible representation with highest weight $\sum_{i=1}^n h_i\omega_i$ at a particular element $g\in G$, put simply one must look at $g$ and determine what the eigenvalues of $g$ are in some fixed representation (usually a defining representation such as your action of $SO(2n)$ on $\mathbb{R}^{2n}$). The logarithms of these eigenvalues can then be expressed in terms involving the weights of the given representation and a certain set of coordinates; these coordinates are the $t_i$ alluded to above.

With that, lets address some of your specific questions in terms of the above background. Since your formula involves hyperbolic sines and cosines, your $t_i$ almost certainly should be the coordinates $t_i$ mentioned before and not the eigenvalues themselves.

Now to the specific element you mention, one is given the $2n$-dimensional representation for this element:

\[
g = \begin{pmatrix}\cos(\alpha)&\sin(\alpha)&0^T_{2n-2}\\-\sin(\alpha)&\cos(\alpha)&0^T_{2n-2}\\0_{2n-2}&0_{2n-2}&Id_{2n-2}\end{pmatrix}
\]

This element is already in a form that gives the eigenvalues of $g$ within a certain representation; its eigenvalues in this representation are $\{\cos(\alpha)\pm i\sin(\alpha),1,\ldots,1\}$. These have logarithms $\{\pm i\alpha,0,\ldots,0\}$. Now one wants to write these logarithms in terms of the weights of this $2n$-dimensional representation being used to express $g$. The full set of weights can be obtained via the method of lowering weights starting at the highest weight, doing so one has the following set of weights for this particular representation:

\[
\pm(\omega_1), \pm(\omega_2-\omega_1), \ldots, \pm(\omega_{n-2}-\omega_{n-3}),\pm(\omega_{n-1}+\omega_{n}-\omega_{n-2}),\pm(\omega_{n-1}-\omega_{n})
\]

To determine our specific coordinates $t_j$ (which ought to be the $t_j$ you use in your formula instead of the eigenvalues) we have alot of choices to make. I think of $\omega_j$ as the coordinate vector with a 1 in its $j^{th}$ position and think of the logarithms of $g$ as the $n$-long vector $v(g) = (i\alpha,0\ldots,0)$ where in general we have arbitrarily chosen one logarithm from each $\pm$ pair (in this way, $v(g)$ now corresponds to the highest weight of this representation evaluated at $g$). My $s_j$ then correspond to the coordinates needed to express $v(g)$ in terms of a weight basis where one weight is chosen from each $\pm$ pair in the list above; i'll go ahead and choose all $+$ signs. So we have:

\[
v(g) = t_1(1,0,\ldots,0) + t_2(-1,1,0,\ldots,0)+\ldots+t_{n-2}(0,\ldots,0,-1,1,0,0)+t_{n-1}(0,\ldots,0,-1,1,1)+t_{n}(0,\ldots,0,1,-1)
\]

So the "coordinates" $t_i$ one should use for this particular $g$ are simply $(t_1,\ldots,t_n) = (i\alpha,0,\ldots,0)$ and these coordinates are what should be used in your formula.

Note that we had alot of choices here, namely about which weight and which logarithm we should choose from each pair and even what order to choose the weights and logarithms in, but in the end each of these possible choices correspond to actions of different elements of $W(G)$ on the character. However, one useful fact about characters is that they are $W(G)$-invariant which is why the particular choices made will not affect your your calculation.