Peter Sarnak believes that integer factorization is in $P$. It is a well-known open problem in TCS to identify the real complexity class of integer factorization. Take a look at this link for Peter Sarnak's lectures where he mentions that he does not believe factoring is not in $P$.

What evidence is there that integer factorization is in $P$ other than the fact that polynomial factorization is in $P$?