What are the most attractive Turing undecidable problems in mathematics? What are the most attractive Turing undecidable problems in mathematics? 
There are thousands of examples, so please post here only the most attractive, best examples. Some examples already appear on the Wikipedia page.
Standard community wiki rules. One example per post please. I will accept the answer I find to be the most attractive, according to the following criteria:


*

*Examples must be undecidable in the sense of Turing computability. (Please not that this is not the same as the sense of logical independence; think of word problem, not Continuum Hypothesis.)

*The best examples will arise from natural mathematical questions.

*The best examples will be easy to describe, and understandable by most or all mathematicians. 

*(Challenge) The very best examples, if any, will in addition have intermediate Turing degree, strictly below the halting problem. That is, they will be undecidable, but not because the halting problem reduces to them. 
Edit: This question is a version of a previous question by Qiaochu Yuan, inquiring which problems in mathematics are able to simulate Turing machines, with the example of the MRDP theorem on diophantine equations, as well as the simulation of Turing machines via PDEs. He has now graciously merged his question here.
 A: A MODULAR SYSTEM $M$ is a finite set of "rules" of the form $ax+b\to cx+d$, 
with $a,b,c,d\in\mathbb{Z}$. If $u,v\in\mathbb{Z}$, then  $u$ is "derivable" from $v$ in $M$ if  one can get from $u$ to $v$ by applying  rules in $M$. For example, the well-known Collatz problem asks whether for all positive integers $u$, 1 is derivable from $u$ in the modular system with the two rules $2x\to x, 2x+1\to 6x+4$. 
The general problem of whether $u$ is derivable from $v$ in a given modular system $M$ is undecidable. (Proved in Borger, "Computability, Complexity and Logic").
A: Rice's Theorem is interesting. It states that only trivial properties of programs are decidable.
http://mathworld.wolfram.com/RicesTheorem.html
A: As I mentioned in the other thread, Matiyasevich's theorem implies that it is undecidable whether a system of Diophantine equations over $\mathbb{Z}$ has a solution (Hilbert's 10th Problem).  I have to mention some related results here: if $\mathbb{Z}$ is replaced by $\mathbb{F}_p[t]$ then the problem is still not decidable, if replaced by $\mathbb{R}, \mathbb{C},$ or $\mathbb{Q}_p$ then the problem is decidable, and if replaced by $\mathbb{Q}$ or $\mathbb{F}_p((t))$ the answer is not known!  (Reference.)  I believe it is not even known whether the answer is yes for some number fields but no for others (which would be truly bizarre).
A: In relation to the undecidability for 3x3 matrices and the answer of 
https://mathoverflow.net/users/1587/john-stillwell that
"The corresponding problem for 2×2 matrices is apparently still open": 
There is a recent proof that the membership for non-singular 2×2 integer matrices is decidable (i.e. for 2x2 integer matrices with nonzero determinant). http://arxiv.org/abs/1604.02303
However in terms of uncedaibiltity the identity problem for 3x3 integer matrices is still open, while the general open problem about the identity matrix was proved to be undecidable for 4x4 matrices over integers , see
Paul C. Bell, Igor Potapov: On the Undecidability of the Identity Correspondence Problem and its Applications for Word and Matrix Semigroups. Int. J. Found. Comput. Sci. 21(6): 963-978 (2010) 
and arxiv.org/abs/0902.1975
solving the long standing open problem see Problem 10.3 in http://press.princeton.edu/math/blondel/solutions.html
Unsolved Problems in Mathematical Systems and Control. Theory, Princeton Univ. Press, 2004.
It also follows that whether a matrix semigroup is a group is undecidable for 4x4 integer matrices.
--- extra comments ------
Finally here are the comments about the importance of the are on matrix products.
Matrices and matrix products play a crucial role in the representation and analysis of various computational processes, i.e., linear recurrent sequences, arithmetic circuits, hybrid and dynamical systems, probabilistic and quantum automat, stochastic games, broadcast protocols, optical systems, etc. Many simply formulated and elementary problems for matrices are inherently difficult to solve even in dimension two, and most of these problems become undecidable in general starting from dimension three or four. One such hard question is the Membership Problem. 
Even algorithmic problems for matrices over SL(2,Z) have many important complexity questions. They are appear in the context of many fundamental
problems from hyperbolic geometry, dynamical systems, Lorenz/modular knots, braid groups, particle physics, high energy physics, M/string theories, ray tracing analysis, music theory, etc.
I would like to also to cite Prof. J. N. Tsitsiklis http://www.mit.edu/~jnt/complex.html  from the webpage "Computational complexity in systems and control" which contains results on computational problems for matrix products:   
"The subject is multifaceted and interesting in many different ways. It can help the practitioner in choosing problem formulations, and in calibrating expectations of what can be algorithmically accomplished. For the systems theorist or the applied mathematician, it raises a variety of challenging open problems that require a diverse set of tools from both discrete and continuous mathematics. Finally, for the theoretical computer scientist, the problems in systems and control theory provide the opportunity to relate abstractly defined complexity classes and specific problems of practical interest."
A: I'm surprised nobody's mentioned the Post correspondence problem.  Like the tiling problem, it seems like something so basic, there has to be some simple way to brute-force it... but no.
A: Let a finite structure be a finite underlying set A 
along with a finite set of finitary operation from A to 
itself.  Such creatures are called algebras in the 
study of universal algebra, and one subarea of study is 
the equational theory of such a creature, i.e. the set 
of universally quantified equations which hold in the 
strucure (e.g. the associative law for semigroups).
The equational theory is said to be finitely based if 
there is a finite set of equations from which one can 
deduce precisely those equations in the equational 
theory.  
A problem raised by Tarski and shown undecidable by 
McKenzie is Tarski's Finite Basis Problem: Given a 
finite structure , determine whether its equational 
theory is finitely based.
Gerhard "Ask Me About System Design" Paseman, 2010.01.12
A: For completeness: Non trivial properties of languages are undecidable.
A: Not mentioned yet, that any computer language extended with non-deterministic features is also Turing computable. 
This is interesting, because it allows the language to be simplified. If the programs operate on objects that are nil or a pair. Then you only need five instructions:


*

*The constant nil

*A pair operator

*A sequence operator, that executes one code fragment after another

*An inverse operator

*A closure operator, which repeats a code fragment zero or multiple times


If you want to construct a piece code of that adds the two values of a pair, then first make something that construct (a - 1, b + 1) from (a, b). Then take the closure. This will generate (a - n, b + n). Finally, pick the value (0, c) and output c. This can be done by using the inverse operator on the pair and nil. 
So, programming is a little bit odd, because you select the right value outside the loop (closure), rather than inside the loop, as in deterministic languages. The advantages is the much more simpler structure. No variables, no recursion, no matching operators (just use the inverse) and no control-structures, except closure.
This makes it a little bit between programming and mathematics. The simpler structure, allows easier mathematical reasoning. So, it might be an idea to convert a program in a deterministic language, to a program of a simplified non-deterministic language, before doing any mathematics on the program.
Lucas
A: In your last criterion, you are essentially asking for a "natural" problem that is nonrecursive, recursively enumerable, and is not complete for the recursively enumerable sets.  Post proved the existence of such problems in Recursively enumerable sets of positive integers and their decision problems, for many-one reductions.  Friedberg and Muchnik proved this also holds for Turing reductions, in separate papers Two recursively enumerable sets of incomparable degrees of unsolvability (solution of Post's problem, 1944) and On the unsolvability of the problem of reducibility in the theory of algorithms.  Whether these are "attractive" is probably determined by whether you like nonconstructive arguments.  For a clear and self-contained exposition of these results, see Kozen's book Theory of Computation.
So this is only a partial answer, and it would still be nice to exhibit a real problem with intermediate degree.
Edit: in the survey Degrees of Unsolvability (which appears to be a chapter of an unpublished Volume 9 of the Handbook of the History of Logic), Ambos-Spies and Fejer state "it is fair to say that every particular c.e. set of natural numbers that has arisen from nonlogical considerations so far is either computable or complete... Thus one could say that the great complexity in the structure of the c.e. degrees arises solely from studying unnatural problems."  This is quite a negative assessment!  On a more positive note, Feferman showed in Degrees of Unsolvability Associated with Classes of Formalized Theories that every c.e. degree arises as the degree of some recursively axiomatizable consistent theory of first-order predicate calculus.
A: There is no algorithm that given a positive integer $K$ can decide if the following concrete Diophantine equation has a solution over non-negative integers:

\begin{align}&(elg^2 + \alpha - bq^2)^2 + (q - b^{5^{60}})^2 + (\lambda + q^4 - 1 - \lambda b^5)^2 + \\
&(\theta + 2z - b^5)^2 + (u + t \theta - l)^2 + (y + m \theta - e)^2 + (n - q^{16})^2 + \\
&((g + eq^3 + lq^5 + (2(e - z \lambda)(1 + g)^4 + \lambda b^5 + \lambda b^5 q^4)q^4)(n^2 - n) + \\
&(q^3 - bl + l + \theta \lambda q^3 + (b^5-2)q^5)(n^2 - 1) - r)^2 + \\
&(p - 2w s^2 r^2 n^2)^2 + (p^2 k^2 - k^2 + 1 - \tau^2)^2 + \\
&(4(c - ksn^2)^2 + \eta - k^2)^2 + (r + 1 + hp - h - k)^2 + \\
&(a - (wn^2 + 1)rsn^2)^2 + (2r + 1 + \phi - c)^2 + \\
&(bw + ca - 2c + 4\alpha \gamma - 5\gamma - d)^2 + \\
&((a^2 - 1)c^2 + 1 - d^2)^2 + ((a^2 - 1)i^2c^4 + 1 - f^2)^2 + \\
&(((a + f^2(d^2 - a))^2 - 1) (2r + 1 + jc)^2 + 1 - (d + of)^2)^2 + \\
&(((z+u+y)^2+u)^2 + y-K)^2 = 0.
\end{align}
A: The Word Problem for groups is undecidable. This is the problem, given a finite group presentation and a word, to decide if that word is the group identity in that presentation. The problem is undecidable because one may encode the Halting problem for Turing machines. Basically, for each Turing machine program, one can construct a group presentation and a word, such that the program halts if and only if that word is the identity. 
A: The Halting Problem, the mother of them all. 
A: The Tiling problem is undecidable. This is the problem, given a finite set of tile types, to determine whether there is an arrangement of them with adjacent sides matching that tiles the plane. The problem is undecidable because the Halting problem for Turing machines reduces to it, in the sense that every Turing machine program corresponds to a tiling problem, which has a tiling if and only if the program fails to halt. Basically, the run of the machine is encoded into the tiling, which can continue as long as the program keeps running.
A: Richardson's theorem says that it is undecidable to tell whether an expression $E$ satisfies $E=0$, where $E$ is generated by $\mathbb{Q}\cup\{\pi,\ln 2,x\}$ and the composition of operations in $\{+,-,\times,\sin,\exp, \mathrm{abs}\}$.
(I thought this deserves its own answer, even if it's given as a comment on this other answer.)
A: My own favorite is that effective first-order theories are in general semi-decidable (ie, recursively enumerable), which follows from the conjunction of Godel's completeness and incompleteness theorems. 
A: As a computer scientist, it would be nice to know if a program contains buffer overflows or deadlocks.
A: Type inference for sufficiently powerful type systems, e.g. System F
A: In control theory, simultaneous stabilization of 3 or more systems is undecidable (necessary and sufficient conditions for simultaneous stabilization of 2 systems are known). This and other stabilization problems are discussed in Blondel and Tsitsiklis, A survey of computational complexity results in systems and control, Automatica, 2000, vol36 n9 p1249--1274, and its references.
A: Given a finite relational language with at least one binary relation, the question of which formulas are finitely satisfiable (i.e. realized in at least one finite structure) is $\Sigma^0_1$ but not computably enumerable (by Trakhtenbrot's Theorem)
A: Here's a nice one: V. D. Blondel, O. Bournez, P. Koiran, C. Papadimitriou, J. N. Tsitsiklis, Deciding stability and mortality of piecewise affine dynamical systems, Theoretical Computer Science, 255: (1-2), pp. 687-696, 2001. (http://www.inma.ucl.ac.be/~blondel/publications/99BBKPT-plstab.pdf)
A: Let $n\geq 3$. Given two embeddings of $S^n$ into $\mathbb{R}^{n+2}$, the problem of determining whether they are equivalent (via a deformation of $\mathbb{R}^{n+2}$) is undecidable (the case $n=2$ is open; for $n=1$ an algorithm exists).
By the way, Bjorn Poonen has a wonderful talk on this topic, titled Undecidability Everywhere.
A: Is a given computable function $f:\mathbb{R}\to\mathbb{R}$ differentiable?
OK, I'll have to (1) clarify what I mean and (2) show it's not a completely trivial consequence of the halting problem.
Part (1):
I have to define computability of $f$. Say that a Turing machine computes a real $x$ if, given any input $n$, it always returns a sequence of $n$ rational numbers, with the $i$th element within $2^{-i}$ of $x$. In other words, it computes the initial part of a Cauchy sequence approximating $x$ to a predetermined accuracy.
Now we can say that a machine $X$ computes $f:\mathbb{R}\to\mathbb{R}$ if for any $x$, you can give it a description of a Turing machine to compute $x$ as an input, and always gives you back another one that computes $f(x)$.
It is impossible to make a machine that takes a description of a machine $X$ to compute $f$, and tells you if the function $f$ is differentiable, i.e. differentiability is undecidable.
But, you say, that's trivial. After all, the machine $X$ we're passing in as argument is, obviously, a machine, so we expect to meet the halting problem. So contrast with:
Part (2):
Integration over an interval is computable.
(I've probably made some typos in the above as it's not my field. So try Computable Analysis by Klaus Weihrauch for more details.)
A: Two open problems in this area that I like:


*

*(related to the Tiling Problem mentioned by Hamkins) Is it undecidable whether a polyomino (as defined e.g. at https://en.wikipedia.org/wiki/Polyomino) tiles a rectangle? If $P$ is a polyomino that tiles a rectangle, let $f(P)$ be the least number of copies of $P$ that are needed to tile a rectangle, and let $p(n)$ be the maximum of $f(P)$ over all polynominoes with $n$ squares that tile a rectangle. If the answer to the question is positive (which is what people in the area believe and is sometimes erroneously claimed to be known), then $p(n)$ grows faster than any recursive function! 

*Let $F(x)=P(x)/Q(x)$ such that $P(x)$ and $Q(x)$ are polynomials with integer coefficients and $Q(0)\neq 0$. Is it undecidable that the Taylor series expansion of $F(x)$ at $x=0$ has a zero coefficient?
A: The emptiness problem for 1-way probabilistic finite state automata is undecidable. (See Condon Lipton Frievalds (sp?).)
A: Say that an algorithm is reliable if given a string $S$ and integer $N$, it either halts and prints $|S|>N$ (meaning that Kolmogorov complexity of the string $S$ is greater than $N$) or does not halt, and never gives a false answer. For example, an algorithm that never halts on any input is reliable.
For any reliable algorithm $A$ there exists an integer $K$ such that for all $N>K$, $A(S,N)$ does not halt on any string (but note that for all strings except a finite set, $|S|>N$ actually holds).
A: Team games, as defined in Bob Hearn's thesis or the book Games, Puzzles, and Computation.   These are games, like bridge, in which there are two teams playing against each other and each team has several players who do not have complete information about the game.  The astounding thing is that even though there are only finitely many game states, it is undecidable to determine whether there is a winning strategy.  This seeming paradox arises because the players do not necessarily know that the game state has returned to a previous state, and a winning strategy can in principle depend on the entire history of the game.  I like this one because it takes some effort even to understand why it it not trivially decidable.
A: Four from Compiler Science:


*

*Does a program ever access an uninitialized variable.

*Do two context free grammars describe the same language.

*Does it make a difference if parameters to a subroutine are passed by reference or by copy-result

*Deadlock determination in parallel programs.


Actually almost every question of the form "Does a program ever do X?" is equivalent to the halting problem.  So the above might be considered too close the the halting problem to be interesting answers ot this question.
A: The problem of distinguishing two manifolds (up to homeomorphism, or even homotopy equivalence Edit: given a triangulation) is undecidable.  This follows from the word problem applied to fundamental groups.
There are similar problems concerning simplicial complexes, e.g., whether a given complex is a triangulation of a manifold.
A: My favourite is related to the Kolmogorov Complexity of a string:

The problem of deciding if a string $s$ is compressible ($K(s) <^? |s| $) is undecidable

A: Well if we're going to give easy ones, then: checking if two real numbers are equal.  As if you needed more reasons to be disturbed by the reals!
A special case of: checking if a vector $v$ in a finite dimensional vector space over the reals is linearly independent of a set of vectors $\{u_i\}$.
(almost equivalently: checking equality (in the sense of extensionality) of $k\geq 2$ bounded integer-valued functions.  the output of such functions can be written as real numbers in $[0,1]$, but you have to have to pad each integer so that you don't accidentally call two different outputs the same real number (due to $0.99\ldots = 1.00$ etc).  How to solve the halting problem: have a function $f(n) = 1$.  Given some arbitrary program/function, nest it in a function $g(n)$ which runs it for $n$ cycles, and outputs $1$ if it halted, $0$ otherwise.  $f$ and $g$ are equivalent iff the program does not halt.)
A: Note that any set of intermediate Turing degree must lie in $L$; so I nominate the least such, with respect to the canonical ordering of $L$.
I suppose this set might have already been mentioned--it's hard to say.
A: The following undecidable problem is natural for engineers in the sense that runtime estimation is an ubiquitous engineering problem associated to (for example) control theory and circuit design.  

Viola's theorem  Given an integer $k$ and Turing machine $M$ promised to be in P, the question "Is the runtime of $M$ of ${O}(n^k)$ with respect to input length $n$ ?" is undecidable.

The proof of this problem's undecidability was given on TCS StackExchange by Emanuele Viola in answer to the question Are runtime bounds in P decidable? 
Background
This question arose in parsing Luca Tevisan's answer on TCS StackExchange to the question Do runtimes for P require EXP resources to upper-bound? … are concrete examples known? (answer: yes and yes).  
The illumination sought in asking/answering this question was a better appreciation/intuition regarding the practical aspects of runtime estimation in the complexity class P, in the sense of runtime estimates that are feasible (that is, require computational resources in P), versus infeasible (that is, require computational resources in EXP), versus formally undecidable (the instance above).
What this problem's undecidability shows us, perhaps, is that some aspects of P are richer and more subtle than is readily appreciated upon first acquaintance.
A: This I think is interesting:  On a finite game board, but with an unbounded number of moves,
games pitting teams against one another, in the presence of imperfect information,
are undecidable.  "Imperfect information" is like that in Bridge (although Bridge has a bounded number of moves).  This result is proved in
Games, Puzzles, & Computation
by Robert Hearn and Erik Demaine, 2009.
A: A variant of the "given a finite simplicial complex, is it the 5-sphere?" problem is the "given a finite simplicial complex, is it it a 6-manifold?".
I find this attractive because, because manifolds are such a basic and fundamental concept, you'd expect we'd be able to recognize one, but in fact we cannot.
This was pointed out by an answer to the question: When are (finite) simplicial complexes (smooth) manifolds?
A: My favorite example is the halting problem for Conway's "FRACTRAN" programming language: given a finite sequence of fractions q1, q2, ...., q_n, does the procedure "starting with a given integer and keep successively multiplying by the first element in the sequence which results in the product still being an integer until none of them do" halt? In fact there is specific sequence of fractions that is quite short which can be interpreted as a Universal machine.
A: Above it was mentioned that, from a general finite group presentation, it is not decidable whether the group is finite. There are actually a bunch of group properties that are similarly undecidable - is it abelian, solvable, simple? But my favourite would be: does it have more than one element!?
A: The recognition problem for compact, simply connected contact manifolds of given dimension $2n-1\geq 11$ (Seidel, 2007).
A contact structure on a $(2n-1)$-manifold is a tangent hyperplane field $\xi$ which can locally be written as $\ker\alpha$ for a 1-form $\alpha$ with $\alpha \wedge (d\alpha)^{n-1}$ non-vanishing. In principle there are finite ways to specify contact manifolds, using symplectic handlebody theory. But there's a simply connected contact manifold $(M_0,\xi_0)$ with the property that, given another, say $(M,\xi)$, the problem of deciding whether it's isomorphic to $(M_0,\xi_0)$ contains an algorithmically-unsolvable word problem for groups. 
If you forget the contact structure, algorithmic recognition is possible (Nabutovsky-Weinberger).
A: The mortality problem for $3\times 3$ matrices: given a finite set $F$ of $3\times 3$
integer matrices, decide whether the zero matrix is a product of members of $F$ 
(with repetitions allowed). This was proved unsolvable by Michael Paterson, 
Studies in Applied Mathematics 49 (1970), 105--107, doi: 10.1002/sapm1970491105.
The corresponding problem for $2\times 2$ matrices is apparently still open.
Edit (11 September 2016): The problem for $2\times 2$ matrices is apparently still open, despite the solution of a seemingly similar problem mentioned in Igor Potapov's answer below.
A: Suppose we specify a group $G$ by a set of relations, e.g. $x_1x_2x_3^{-1} = 1$.  Then, the problem of determining if $G$ is finite or not is undecidable.
A: For your "challenge" question, note that it is extremely hard to construct examples of problems of degree strictly less than the halting problem. In fact this was a question open for some years under the name of Post's problem. It was finally solved by the invention of the "finite injury method" which gave many examples of such problems. However I do not know of any naturally formulated problem with degree strictly less than the halting problem.
A: From what I've been told, it is impossible to decide whether (in general) a complex holomorphic function has a zero at the origin.  Similarly, it is undecidable whether certain holomorphic functions have double zeros or zeros just "really" close to each other.  [It would really be interesting if this had some effect on things like the Riemann hypothesis or BSD.]
A: The rule 110 is also a cute one.
A: As an extension of Matiyasevich's theorem, there is a constant $c$, and a Diophantine equation on $n+1$ parameters and $m$ variables:
$$D(K,a_1,a_2,\cdots,a_n,x_1,x_2,\cdots,x_m)=0$$
including a parameter $K$ such that, for each $K\ge c$, there is no algorithm to determine whether the Diophantine equation has an infinite number of solutions or a finite number of solutions.
A similar result holds by replacing "infinite" with "even" and "finite" with "odd."
Here, the Diophantine equation has an infinite (resp. even) number of solutions iff the $K$'th bit of $\Omega$ is $1$ (where $\Omega$ is Chaitin's Constant.)
See, e.g., Ord and Kieu, "On the existence of a new family of equations for $\Omega$" (link)
