I agree with Tim that calling Pursuing Stacks a "letter to Quillen" is erroneous, especially as Quillen never replied.
Grothendieck also wrote: "This is written in English in response to a correspondence in English." At one stage he planned more volumes in French but it seems got diverted from this. I hope the following will be of help, in addition to David Robert's answer, to put the situation with regard to models for homotopy theory in context.

In a letter dated 02/05/1983 Alexander Grothendieck
wrote to me: "Don't be surprised by my supposed efficiency in
digging out the right kind of notions--I have just been following,
rather let myself be pulled ahead, by that very strong thread
(roughly: understand non commutative cohomology of topoi!) which I
kept trying to sell for about ten or twenty years now, without
anyone ready to ``buy'' it, namely to do the work. So finally I got
mad and decided to work out at least an outline by myself."

But this question is about the homotopy theory of categories and the related question of "Why simplicial sets?", for which see also the good answers in March 2011 to Is there a high-concept explanation for why "simplicial" leads to "homotopy-theoretic"?.

Dan Kan's first contribution to combinatorial homotopy was in terms of cubical sets. When he went to Princeton the disadvantages of cubical sets were found: cubical groups did not satisfy the extension property, and the geometric realisation of the cartesian product of cubical sets had the wrong homotopy type, while as shown by Moore and Milnor respectively, the situation was fine for simplicial sets. So they did not attempt to refine the cubical theory. In my doctoral studies at Oxford, 1956-59, the exposition was all simplicial, especially when Michael Barratt came back from Princeton in 1957.

However the 1960 book by Hilton and Wylie on algebraic topology was cubical, as were some 1962 notes of Federer from Brown University, a later book by Massey, and cubical sets continued to be found useful in various places. Our 2011 book on "Nonabelian algebraic topology" is almost entirely cubical, because of its emphasis on the use of higher homotopy Seifert-van Kampen theorems.

My 1965 intuition for using cubical sets was based on generalising the van Kampen theorem to higher dimensions. It seemed entirely reasonable that the above diagram could be expressed as: the big square is the composition of the little squares. C. Ehresmann's 1965 book on "Categories structuree" gave a definition of double categories which expressed this nicely. Indeed, I have answered this on mathoverflow as using matrix notation where $(a_{ij})$ denotes a composable array and $[a_{ij}]$ denotes the composite. So one has an easy definition of the $n$-fold cubes of the nerve of an $n$-fold category, except that there seems currently no name for the cubical type geometry underlying an $n$-fold category. Note that composable sequences of morphisms in a category are used in describing the nerve of a category, but it seems more difficult, at least for me, to define multiple compositions in simplicial or globular terms, although the simplicial nerve of an $n$-fold category is easily defined as an $n$-fold simplicial set.

By contrast, the singular cubical complex of a space, or filtered space, is ideally suited for the description of multiple compositions, using an array notation. I have already explained this in answer to this mathoverflow question.

So in considering what category in which to work the question of "what should be adequacy and convenience?" is crucial. It is as reasonable to ask this for combinatorial models of homotopy theory as it was in 1963 to ask it for categories for topology in my paper Ten topologies.

A property that was also required for the conjectured proof of a putative higher van Kampen theorem using homotopy classes of maps was the notion of "commuting cube", and that "any composition of commuting cubes is commutative". Chris Spencer and I found that the notion of "connection" for a double groupoid was good for this, and that it allowed an equivalence between crossed modules and single pointed edge symmetric double groupoids with connections. Then Philip Higgins and I found in 1974 the construction of the homotopy double groupoid of a pair $(X,A,x)$ of pointed spaces using homotopy classes of maps $I^2 \to X$ which take the edges to $A$ and the vertices to $x$. This gave the first homotopy fundamental double groupoid, which enabled the proof of a 2-dimensional van Kampen theorem, including the usual theorem for the fundamental group as a special case, not just an implication.

There are grounds for suggesting that simplicial sets are convenient, but are not entirely adequate, since they cannot easily express multiple compositions. On the other hand, cubical sets with connections are adequate for this test, but not entirely convenient! Andy Tonks proved that cubical groups with connection are Kan complexes. One reason for inconvenience is that although they have been shown to form a strict test category in the sense of Grothendieck, in the paper given here, the geometric realisation of the categorical product is only of the homotopy type of the product of the realisations, not actually homeomorphic to the product as in the case of simplicial sets. The latter homeomorphism property implies that in the right convenient category, the geometric realisation of a simplicial group is a topological group.

The cubical setup is also not sufficient to describe the geometry underlying $n$-fold categories, and indeed there seems currently no name for such a structure in which cubes have different types of faces in different directions. Yet Grothendieck remarked to me on my affirming Loday's theorem, that (strict) $n$-fold groupoids model weak homotopy $n$-types: "That is absolutely beautiful!"

So it as well not to assume we have the final story, and to investigate options!

January, 2015: This answer is related to my answer to

https://math.stackexchange.com/questions/1112107/why-does-seifert-van-kampen-not-hold-with-n-th-homotopy-groups/

November, 2016 There is more discussion in this preprint Modelling and Computing Homotopy Types: I.

Champs Affines,Bertrand Töen. $\endgroup$1more comment