This topic is pretty well-known in studies of function spaces and when they satisfy the appropriate adjointness condition (where the functor $- \times X: \mathbf{Top} \to \mathbf{Top}$ is left adjoint to $Cont(X, -)$; we call such spaces $X$ exponentiable). When one tries to apply the general adjoint functor theorem to construct a right adjoint $Cont(X, -)$, one is led pretty quickly to the condition that there should be a coarsest admissible topology, in order to get the correct topology on $Cont(X, Y)$).
Stefan Geschke made a very pertinent comment that (for Hausdorff spaces at least) it is local compactness of $X$ which is the decisive factor for this condition; more exactly, when $X$ is Hausdorff, $X$ is exponentiable iff it is locally compact.
This paperpaper by Escardó and Heckmann gives a general analysis, showing that for more general (possibly non-Hausdorff) $X$, the decisive condition is something called core-compactness (if $X$ is Hausdorff, then core-compactness is equivalent to local compactness). Probably the most elegant way of formulating it is that $X$ is core-compact iff the topology of $X$ is a continuous lattice.
To be more explicit: For open subsets $U$ and $V$ of a topological space $X$, let us write $V\ll U$ to mean that any open cover of $U$ admits a finite subcover of $V$; this is read as $V$ is relatively compact under $U$ or $V$ is way below $U$. We say that $X$ is core-compact if for every open neighborhood $U$ of a point $x$, there exists an open neighborhood $V$ of $x$ with $V\ll U$. In other words, $X$ is core-compact iff for all open subsets $V$, we have $V = \bigcup \{ U | U\ll V \}$. The theorem is that $X$ is exponentiable iff it is core-compact.
For what it's worth, I did a little write-up at the $n$-Category Café some years back, giving a concrete counterexample much the same as Eric's but with $X$ the space of rationals $\mathbb{Q}$; it can be found here. This example was based on my reading of the paper mentioned above; it turns out that if $X$ is not core-compact, then a counterexample (to the assertion that the intersection of admissible topologies is still admissible) can be always be found by taking $Y$ to be Sierpinski space $\mathbf{2}$ (as in Eric's example).