The Maple 18 code 

    A := eval(convert(hypergeom([-a, k-a-1/2], [k-a], y), FPS, y), y = 4*x*t*(-t*x+1)) assuming k::posint:
    int(A*t^(-a)*(1-t)^N*(-t*x+1)^(-a), t = 0 .. 1) assuming a>0,a<1,N::posint,x>0,x<1;

outputs the integral under consideration as the series
$$\Gamma(N+1)\times$$
 $$\sum\limits_{i=0}^\infty \frac{\mathop{\rm pochhammer}(-a,i)x^{i}2^{2i}{\mbox{$_2$F$_1$}(- i+a,1-a+ i;-a+ i+2+N;x)}}{\mathop{\rm pochhammer}(k-a,i)\Gamma(-a+i+2+N)\mathop{\rm pochhammer}(k-a-1/2,i)^{-1}i!},$$
 where $\mathop{\rm pochhammer}$ is described [here](http://www.maplesoft.com/support/help/Maple/view.aspx?path=pochhammer). See [here](https://www.dropbox.com/s/r1dhmwtr8fvbanv/hypergeom.pdf) for the Maple output.