Here's an elementary argument proving that $I$ is infinite.

Claim. $p^{17} \notin I$ for all primes $p > 19$.

Proof. Suppose that $p^{17}=\frac{N}{d(N)}$ for some $N$. Write $N=p^{17+k}n$ where $p$ does not divide $n$. Then, $d(N)=(18+k)d(n)$, and so $p^kn=(18+k)d(n)$. Since $p>19$ and $n \geq d(n)$ this can only hold if $k=0$. But now $\frac{n}{d(n)}=18$, which Greg Martin has shown is impossible.

Here's an elementary argument proving that the set of numbers $I$ that fail the conjecture is infinite.

Claim. $p^{17} \in I$ for all primes $p > 19$.

Proof. Suppose that $p^{17}=\frac{N}{d(N)}$ for some $N$. Write $N=p^{17+k}n$ where $p$ does not divide $n$. Then, $d(N)=(18+k)d(n)$, and so $p^kn=(18+k)d(n)$. Since $p>19$ and $n \geq d(n)$ this can only hold if $k=0$. But now $\frac{n}{d(n)}=18$, which Greg Martin has shown is impossible.

Here's an elementary argument proving that $I$ is infinite.

Claim. $p^{17} \notin I$ for all primes $p > 17$.

Proof. Suppose that $p^{17}=\frac{N}{d(N)}$ for some $N$. Write $N=p^{17+k}n$ where 17 does not divide $n$. Then, $d(N)=(18+k)d(n)$, and so $p^kn=(18+k)d(n)$. Since $p>18$ and $n \geq d(n)$ this can only hold if $k=0$. But now $\frac{n}{d(n)}=18$, which Greg Martin has shown is impossible.

Here's an elementary argument proving that $I$ is infinite.

Claim. $p^{17} \notin I$ for all primes $p > 19$.

Proof. Suppose that $p^{17}=\frac{N}{d(N)}$ for some $N$. Write $N=p^{17+k}n$ where $p$ does not divide $n$. Then, $d(N)=(18+k)d(n)$, and so $p^kn=(18+k)d(n)$. Since $p>19$ and $n \geq d(n)$ this can only hold if $k=0$. But now $\frac{n}{d(n)}=18$, which Greg Martin has shown is impossible.