Hilbert's 23 problems, ten of which were presented at the 1900 ICM in Paris, are too famous for any mathematician to not know. If one reads the descriptions of the problems in Hilbert's paper, one realizes that some questions are concrete whereas the others are stated somewhat vaguely. The 24th problem that I will quote below definitely falls into the latter category.

It seems that there was a 24th problem which was "cancelled". The following is from an article that appeared in American Mathematical Monthly in 2003.

Let me begin by presenting the problem itself. The twenty-fourth problem belongs to the realm of foundations of mathematics. In a nutshell, it asks for the simplest proof of any theorem. In his mathematical notebooks [38:3, pp. 25-26], Hilbert formulated it as follows (author's translation):

The 24th problem in my Paris lecture was to be: Criteria of simplicity, or proof of the greatest simplicity of certain proofs. Develop a theory of the method of proof in mathematics in general. Under a given set of conditions there can be but one simplest proof. Quite generally, if there are two proofs for a theorem, you must keep going until you have derived each from the other, or until it becomes quite evident what variant conditions (and aids) have been used in the two proofs. Given two routes, it is not right to take either of these two or to look for a third; it is necessary to investigate the area lying between the two routes. Attempts at judging the simplicity of a proof are in my examination of syzygies and syzygies [Hilbert misspelled the word syzygies] between syzygies [see Hilbert [42, lectures XXXII-XXXIX]]. The use or the knowledge of a syzygy simplifies in an essential way a proof that a certain identity is true. Because any process of addition [is] an application of the commutative law of addition etc. [and because] this always corresponds to geometric theorems or logical conclusions, one can count these [processes], and, for instance, in proving certain theorems of elementary geometry (the Pythagoras theorem, [theorems] on remarkable points of triangles), one can very well decide which of the proofs is the simplest. [Author's note: Part of the last sentence is not only barely legible in Hilbert's notebook but also grammatically incorrect. Corrections and insertions that Hilbert made in this entry show that he wrote down the problem in haste.]

The paper I linked above discusses the history and the role of Hilbert's problems and I think is worth reading. Most of mathematical logic, as we know it right now, did not exist when this question was asked and you can simply disregard the question by saying "this is not a mathematical question". On the other hand, the same could be said about the second problem on the consistency of arithmetic today, if mathematicians did not develop the necessary tools to deal with this problem.

My point is that one might be able to answer Hilbert's 24th problem if one finds the "correct" statement of the problem. With our current knowledge and understanding of mathematical logic, can we define a criteria for a proof to be "simple"? Have there been any attempts to define such a notion? Should "simple" merely mean "short"?

In Thiele's article, you can find some quotations in Section 5 but they do not really give any useful information about how Hilbert perceived the word "simple". Having stumbled upon this article only today, I admit that I have not searched for other articles yet. So I would also appreciate being directed to other books and articles on this cancelled 24th problem.

reverse mathematicsfalls into this scheme. I say "arguably" because reverse mathematics is concerned only with the axioms used in a proof, not the length or other structure of the proof, so the whole text suggests this isn't what he meant (hence my comment rather than an answer). Still, it's interesting to note. $\endgroup$ – Noah Schweber Oct 15 '15 at 7:15shouldbe, which is subjective. It seems that you can, for example, discuss Turing machines on MO but you cannot discuss the Church-Turing thesis. $\endgroup$ – Burak Oct 17 '15 at 20:30