I am trying to show that $\sum\limits_{j=0}^{k-1}(-1)^{j+1}(k-j)^{2k-2} \binom{2k+1}{j} \ge 0$. This inequality has been verified by computer for $k\le40$.

Some clues that might work (kindly provided by Doron Zeilberger) are as follows:

1. Let $Ef(x):=f(x-1)$, let $P_k(E):=\sum_{j=0}^{k-1}(-1)^{(j+1)}*\binom{2*k+1}{j}*E^j$;

2. These satisfy the inhomogeneous recurrence 
$P_k(E)-(1-E)^2*P_{k-1}(E)=\text{'some Binomial In E'}$;

3. The original sum can be expressed as $P_k(E)x^{(2*k-2)} | x=k $;

4. Try to derive a recurrence for $P_k(E)x^{(2*k-2)}$ before plugging-in
$x=k$ and somehow use induction, possibly having to prove a more general statement 
to facilitate the induction.

Unfortunately I do not know how to find a recurrence such as suggested in 4.