The Maple 18 code
A := eval(convert(hypergeom([-a, k-a-1/2], [k-a], y), FPS, y), y = 4*x*t*(-t*x+1)) assuming k::posint:
int(A*t^(-a)*(1-t)^N*(-t*x+1)^(-a), t = 0 .. 1) assuming a>0,a<1,N::posint,x>0,x<1;
outputs the integral under consideration as the series $$\Gamma(N+1)\times$$ $$\sum\limits_{i=0}^\infty \frac{\mathop{\rm pochhammer}(-a,i)x^{i}2^{2i}{\mbox{$_2$F$_1$}(- i+a,1-a+ i;-a+ i+2+N;x)}}{\mathop{\rm pochhammer}(k-a,i)\Gamma(-a+i+2+N)\mathop{\rm pochhammer}(k-a-1/2,i)^{-1}i!},$$ where $\mathop{\rm pochhammer}$ is described here. See here for the Maple output.
