With some effort (the lower integration limit requires care) I found this answer for the definite integral:
$$I(x)=\int\limits_{\delta}^{x} \left(y-\delta\right)^{-\frac{1}{ \epsilon+3}} y^{-\frac{\epsilon+4}{\epsilon+3}}dy=\frac{ (-2/\delta)^{\frac{2}{{\varepsilon}+3}} \pi ^{3/2} }{\sin \left(\pi\frac{{\varepsilon}+2}{{\varepsilon}+3}\right)\Gamma \left(\frac{{\varepsilon}+1}{2 {\varepsilon}+6}\right) \Gamma \left(\frac{\varepsilon+4}{{\varepsilon}+3}\right)}$$
$$\qquad\qquad+\;\delta^{-1}({\varepsilon}+3) x^{-\frac{1}{{\varepsilon}+3}} (x-{\delta})^{\frac{{\varepsilon}+2}{{\varepsilon}+3}} \, _2F_1\left(1,\frac{{\varepsilon}+1}{{\varepsilon}+3};\frac{{\varepsilon}+2}{{\varepsilon}+3};\frac{x}{{\delta}}\right),$$
for $\delta\equiv \theta-1>0$ and $x>\delta$, $\varepsilon>0$. Each of the two terms on the right-hand-side is complex, but the imaginary parts cancel.
Here is a Mathematica command to test this result against a numerical evaluation of the integral:
Plot[{((-4)^(1/(3 + eps)) delta^(-(2/(3 + eps))) [Pi]^(3/2)
Csc[((2 + eps) [Pi])/(3 + eps)])/( Gamma[(1 + eps)/(6 + 2 eps)] Gamma[
1 + 1/(3 + eps)]) + ((3 + eps) x^(-(1/(3 + eps))) (-delta + x)^((
2 + eps)/(3 + eps))
Hypergeometric2F1[1, (1 + eps)/(3 + eps), (2 + eps)/(3 + eps), x/
delta])/delta,
NIntegrate[ y^(-(eps + 4)/(eps + 3))*(y - delta)^(-1/(eps + 3)), {y, delta,
x}]}, {x, delta, 2}]
with this output for $\varepsilon=0.2,\delta=0.3$ (two indistinguishable curves)