In the eighties, Grothendieck devoted a great amount of time to work on the foundations of homotopical algebra.
He wrote in "Esquisse d'un programme": "[D]epuis près d'un an, la plus grande partie de mon énergie a été consacrée à un travail de réflexion sur les fondements de l'algèbre (co)homologique non commutative, ou ce qui revient au même, finalement, de l'algèbre homotopique." (Beginning of section 7. English version here: "Since the month of March last year, so nearly a year ago, the greater part of my energy has been devoted to a work of reflection on the foundations of non-commutative (co)homological algebra, or what is the same, after all, of homotopic[al] algebra.)
In a letter to Thomason written in 1991, he states: "[P]our moi le “paradis originel” pour l’algèbre topologique n’est nullement la sempiternelle catégorie ∆∧ semi-simpliciale [he means the simplex category, of course], si utile soit-elle, et encore moins celle des espaces topologiques (qui l’une et l’autre s’envoient dans la 2-caégorie des topos, qui en est comme une enveloppe commune), mais bien la catégorie Cat des petites caégories, vue avec un œil de géomètre par l’ensemble d’intuition, étonnamment riche, provenant des topos."
If $Hot$ stands for the classical homotopy category, then we can see $Hot$ as the localization of $Cat$ with respect to functors of which the topological realization of the nerve is a homotopy equivalence (or equivalently a topological weak equivalence). This definition of $Hot$ still makes use of topological spaces. However, topological spaces are in fact not necessary to define $Hot$. Grothendieck defines a basic localizer as a $W \subseteq Fl(Cat)$ satisfying the following properties: $W$ is weakly saturated; if a small category $A$ has a terminal object, then $A \to e$ is in $W$ (where $e$ stands for the trivial category); and the relative version of Quillen Theorem A holds. This notion is clearly stable by intersection, and Grothendieck conjectured that classical weak equivalences of $Cat$ form the smallest basic localizer. This was proved by Cisinski in his thesis, so that we end up with a categorical definition of the homotopy category $Hot$ without having mentionned topological spaces. (Neither have we made use of simplicial sets.)
I personnally found what Grothendieck wrote on the subject quite convincing, but of course it is a rather radical change of viewpoint regarding the foundations of homotopical algebra.
A related fact is that Grothendieck writes in "Esquisse d'un programme" that "la "topologie générale" a été développée (dans les années trente et quarante) par des analystes et pour les besoins de l'analyse, non pour les besoins de la topologie proprement dite, c'est à dire l'étude des propriétés topologiques de formes géométriques diverses". ("[G]eneral topology” was developed (during the thirties and forties) by analysts and in order to meet the needs of analysis, not for topology per se, i.e. the study of the topological properties of the various geometrical shapes." See the link above.) This sentence has already been alluded to on MO, for instance in Allen Knutson's answer there or Kevin Lin's comment there.
So much for the personal background of this question.