Does X(13) have potentially good reduction at 13? The complete level modular curve $X(p)$ does not have potentially good reduction at $p$ for any $p \neq 2,3,5,7,13$ because then there are cusp forms on $X_0(p)$ showing up in the cohomology of $X(p)$, whose associated Galois representations have unipotent local monodromy at $p$, which contradicts potentially good reduction.
$X(p)$ clearly does have good reduction at $p$ for $p=2,3,5$ because it has genus $0$ and a rational point.
$X(7)$ nonobviously has potential good reduction at $p=7$ where $X(7)$, the Klein quartic curve in $\mathbb P^2$, degenerates to a smooth hyperelliptic curve. See, for instance, Elkies's paper on the Klein quartic.
So $p=13$ is the only remaining case. What's the answer there?
A good first step might be to try to answer the question for $X_1(13)$, which only has genus $2$.
 A: If there were a newform which didn't have potentially good reduction and whose associated abelian variety were isogenous to a subvariety of $Jac(X(p))$ then the associated local automorphic representation would have to be a twist of the Steinberg. The character that one is twisting by would have to have conductor 1 or $p$ (otherwise the conductor of the local automorphic rep at $p$ would be bigger than $p^2$) and there's a global character which looks like this locally on inertia; untwisting by this global char shows that the twisted modular form contributes to the cohomology of $X_0(p)$. So if $X_0(p)$ has genus 0 then $Jac(X(p))$ has potentially good reduction at $p$. I don't think this quite proves that $X(p)$ has good reduction at $p$ but it answers François' question at least. The implicit claim in the question that $X_0(p)$ has good reduction at $p$ because it has genus 0 isn't quite a complete argument: twists of the projective line can have bad reduction at some primes, but Will is of course OK because $X_0(p)$ has cusps and hence points.
A: Regarding Will's question on $X_1(13)$:
An equation of this genus 2 curve is
$$y^2 = x^6 - 2 x^5 + x^4 - 2 x^3 + 6 x^2 - 4 x + 1.$$
The polynomial on the right, when reduced mod 13, has a triple root at $-3$ and three simple roots. Over ${\mathbb Q}_{13}$, the roots reducing to $-3$ are of the form $-3+\theta_j$ with $v_{13}(\theta_j) = \frac{1}{3}$ and
$v_{13}(\theta_i - \theta_j) = \frac{1}{3}$ when $i \neq j$. This implies
that over a sufficiently ramified finite extension of ${\mathbb Q}_{13}$
(ramification index divisible by 6 is sufficient), the stable model
of $X_1(13)$ will have special fiber consisting of two elliptic curves
meeting transversally in one point (if my computation is correct,
then both curves have $j$-invariant zero). This in turn means that
the curve $X_1(13)$ has bad reduction over any extension of
${\mathbb Q}_{13}$, but its Jacobian $J_1(13)$ has good reduction
over sufficiently ramified extensions (its reduction is the product of
the two elliptic curves and hence an abelian variety). 
So this is a case of (potentially) "mildly bad reduction".
A: Regarding Will's question on $X(13)$, it follows from Michael Stoll's answer and the following lemma that $X(13)$ does not have potentially good reduction.
Lemma. Let $X\to Y$ be a finite morphism of smooth projective geometrically connected curves over a number field $K$. Suppose that $Y$ has non-zero genus. If $X$ has good reduction over $O_K$, then $Y$ has good reduction over $O_K$.
Proof. This follows from the existence of Neron models for hyperbolic curves (not only abelian varieties); see Corollary 4.7 in Liu-Tong http://arxiv.org/abs/1312.4822 . QED
One now concludes as follows. Suppose that $X(13)$ has good reduction everywhere over some number field $K$. Replacing $K$ by a finite field extension if necessary, there  is a (natural) finite morphism $X(13)\to X_1(13)$. Since  $X_1(13)$ has genus two, it follows from the above lemma  that $X_1(13)$ has good reduction over $O_K$. This contradicts Stoll's answer.
