$\newcommand{\la}{\lambda}\newcommand{\Ga}{\Gamma}$Write \begin{equation} I(\la)=\int_0^\infty dx\,f(x)J(\la x), \end{equation} where \begin{equation} J(z):=\int_0^1 dt\,e^{i z t^a}=\frac b{z^b}K(z), \end{equation} \begin{equation} K(z):=\int_0^z du\,u^{b-1}e^{i u}, \end{equation} $b:=1/a\in(0,1)$. Note that \begin{equation} K(z)\to\int_0^\infty du\,u^{b-1}e^{i u}=(-i)^b\Ga(b) \end{equation} as $z\to\infty$. Also, since $u^{b-1}$ decreases to $0$ as $u$ increases from $0$ to $\infty$, we have $|K(z)|\le C$ for some real $C>0$ and all real $z\ge0$. So, letting $\la\to\infty$, by dominated convergence we get \begin{equation} \la^b I(\la)=\int_0^\infty dx\,f(x)\frac b{x^b}K(\la x) \\ \to (-i)^b\Ga(b+1)\int_0^\infty dx\,f(x)/x^b, \end{equation} so that \begin{equation} I(\la)\sim(i\la)^{-b}\,\Ga(b+1)\int_0^\infty dx\,f(x)/x^b. \end{equation}