What is the geometry of an undecidable diophantine equation? As an arithmetic algebraic geometer of the highest moral fiber, I am trained to look at Diophantine equations in terms of the geometry of the corresponding scheme. For instance, if the Diophantine equation comes from a curve, I know that I should compute the genus, and do various things depending on if the genus is $0$, $1$, or $\geq 2$.
But I know very little about what the limits of this geometric approach are. I only know that there exist undecidable Diophantine equations, or families of Diophantine equations. I do not know what their geometry is like!
Do undecidable Diophantine equations, or families of equations, have interesting geometric properties? Can we compute basic geometric invariants like the Hodge diamond, Kodaira dimension, etc.? Are they pathological in every way, or do some of them have properties that might give a naive geometer hope about finding solutions? What would Noam Elkies try to do if he were asked to solve them and did not know they were undecidable, and why would he be stymied?
 A: You have a typical recursively enumerable set S of integers, and a set X of lattice points cut out by a multivariate polynomial. We are talking about S being the projection (onto one axis) of X. Given that S can be "pretty bad", can one say anything except that X is "presumably worse"? None of the interest lies in any finite segment of S.
To put it another way, possibly more interesting to geometers, the analogy with constructible sets fails here. There is some hint in the history that Hilbert was misled by elimination theory into thinking that Godel's incompleteness theorem (or suchlike) couldn't be the case. (I'm not putting this very well, and it is disrespectful in a way to Hilbert.)  Anyway algebraic geometers "know" that projection doesn't really innovate that much in the way of geometry, and logicians "know" the precise opposite in terms of logic. So at the very least the intuitions are in tension.
(Maybe the mistake of thinking that "geometry of undecidable diophantine systems" was a kind of impossible object, and therefore all r.e. sets would turn out to be recursive, is a mistake intelligent enough to attribute to Hilbert. As a kind of Fundamental Theorem of Proof Theory.)
A: Though I don't have a full answer to your question, the following remarks may help.
Let's distinguish between (1) explicit examples of systems of Diophantine equations that are known to be undecidable, and (2) a system of Diophantine equations that has the property of being undecidable, whether we know it or not.
Regarding number (1), Jones has written down some explicit examples.  The main practical difficulty with computing any invariants of these examples is their size.  The basic example in the paper has 28 variables and degree $5^{60}$, which can be rewritten as a system in 58 variables and degree 2.  My guess is that any Groebner basis algorithm would crash on this example, not because of any particular pathology, but because of its size.  However, maybe I'm wrong.  The example is written out explicitly so you could try playing with it yourself.
Regarding number (2), although I'm not aware of any precise theorems to this effect, I think that the intuition of most logicians and computability theorists is that "most" systems of Diophantine equations are undecidable.  (This is related to the intuition that "most" computably enumerable sets are not computable.)  If this intuition is correct, then undecidable Diophantine equations aren't "special"; it's just the reverse—the decidable ones are the ones that are special.  Then your question is not very different from the question, what is the geometry of a "random" set of Diophantine equations?  So it becomes more of a question in computational commutative algebra than a question in computability theory.
A: This is just a long comment, rather than an answer to Will's question. There are "standard" algorithmically unsolvable problems in combinatorial group theory: Word problem, conjugacy problem, triviality problem for the group, isomorphism problem for a pair of groups, etc. The most fruitful line of arguments relating logic to geometry in group theory is to find geometric conditions on finitely-presented group(s) which are sufficient for solvability of these problems. It turned out that, appropriately defined hyperbolicity is a geometric condition implying algorithmic solvability of all the "standard" problems. Furthermore, it turned out that hyperbolicity is, probabilistically speaking, "generic". Moreover, one of the oldest algorithms for solving word problem (Dehn's algorithm/linear isoperimetric inequality) is one of many equivalent definitions of hyperbolicity. In view of this, I would be looking for algebro-geometric features which are sufficient for algorithmic solvability of Diophantine equations, rather than the other way around. However, since I am neither a number-theorist nor a logician, I do not know what they could be. 
