Edit October 11, 2017
While rereading an article of McLarty's cite below, I just stumbled over a passage highly relevant to the OP, and I think it will be useful to the opening poster. Lest I forget it, I'll quickly add it to the thread, without lengthily giving background for MacLarty's quite advanced article.
How the new addition relates to this thread.
This addition nicely corresponds both with Thomas Rot's comment, which first mentioned ETCS, and also with
"It is true that at the start of XX century, there was a big hope that set theory may provide a unifying framework to the whole of mathematics. This predominant position is perhaps less clear nowadays. It has been challenged by category theory on one hand. You may say that arrows are an abstraction for the concept of functions. These arrows have to map objects to objects, so there is still a need for an abstraction to sets."
[source is second paragraph of this]
Incidentally, to repeat myself, one should not fail to mention that the formulation 'arrows have to map objects to objects' in loc. cit. is, while of course not technically false, an unusual thing to say. The 'arrow's of category theory are something like a 'logical primitive', or a 'sort', or whatever similar word you allow; the arrows are syntax, and proudly so, in particular they need not be interpreted as maps, at least not if by the word 'map' you mean the set-theoretic kind of 'map'(='functional graph'='right-unique binary relation'). Also, more seriously, 'map objects to objects' can look to inexperienced eyes like 'the domain of an arrow contains things called 'objects' and these are being mapped by the arrow', which is not the usual interpretation: the objects are equal to the domain and codomain, respectively.
Enough of the usage guide. Coudy is of course right in emphasizing that there is both a philosophical and historical motivation behind the notion of 'arrow's, and of course, historically, most arrows in early category theory were and are interpreted as functions.
This is what McLarty's passage sheds more light on.
The addition proper.
The source is
Colin McLarty: Exploring Categorical Structuralism. Philosophia Mathematica 3(12), 2004, pp. 37-53.
A scan of the passage is (a text-format transcription follows below; color added)
and, having recently been reminded that one should, if possible, make passages available also in text-format to help visually-impaired readers, a verbatim transcription of the above passage now follows:
3. Presupposition
Hellman follows Feferman [1977] saying categorical foundations presuppose a theory of sets and functions. More precisely: 'There is frank acknowledgement that the notion of function is presupposed, at least informally, in formulating category theory' (p. 133). What does it mean to presuppose something informally? For one thing it means the axioms do not formally presuppose any such thing. If thy did, Feferman or Hellman could show where they do it. It turns out to mean the axioms are motivated by an informal idea of function. [emphasis emphasized; but please note that at this point McLarty is still in the process of repeating what Hellman imputed the axioms to be motivated by, an imputation with which he seems to politely disagree] Certainly ETCS is motivated by an informal ideal of sets and functions. [note the emphasis on ETCS conveyed by the 'Certainly'; McLarty seems to agree that ETCS is so motivated, in particular since ETCS's axiom are more 'assertory' than the general 1-category-axioms] Hellman (pp. 134-135) gives reasons why this is illegitimate, at least in the case of categorical axioms, which I cannot entirely understand. It may hang on his belief that 'category theory-at least as presented in axioms-if "forma" or "schematic": unlike the axioms of set theory, its axioms are not assertory (p. 135). But this confuses the general category axioms with specific categorical foundations such as ETCS. I discuss this further in the next section.
Again, no background for McLarty's article is given here; to understand it, you'll have to study the literature. I end with adding the beginning of Lavere's PNAS 52 article; this seems a harmonious conclusion to this addition to this answer, since in a sense, in Lawvere's article, purely categorical foundations, and set-theoretic sets-and-mappings-foundations, are being reconciled.
Former version of this answer: footnotes to (a comment to) Gerald Edgar's answer.
This is mostly a historical footnote to answers that others have already given. Except for the yellow, which is to highlight what I think makes this contribution 'on-topic' for this thread, the colors in the translation used are not intended to mean anything (which is also rather obvious), the green-blue-alternation is rather to facilitate reading. (And the gray is for what I think needs no translation. And the 'redded' footnote is a footnote von Neumann makes in a part of the text that I leave out, so I do not translate this either; it would lead too far afield: the point of all of this is to give historical background to what Gerald Edgar in effect already remarked: that von Neumann about a century ago wrote he preferred functions over sets to axiomatize set theory. And this is not meant as an 'ipse dixit', i.e. not meant to say 'functions are the better foundation for set theory because von Neumann said so', rather as a historical background and an experiment in communication (in particular the colors in the side-by-side translation).
Strictly speaking this is not an answer to the OP's question, who asked about axiomatizing mathematics, not 'only' set theory, and 'axiomatizing mathematics' has never been done by anyone.
Also, I consider my answer to be a mere piece of historical background to the accepted answer, which is more modern and in touch with contemporary set theory. It is rather a long footnote to what Gerald Edgar and Francois Ziegler already mentioned about von Neumann, and an exercise in using the wonders of electronic writing media.
First, for readers' convenience, a relevant passage from the Blass--Grurevich article referenced by Francois Ziegler:
My knowledge of the history of mathematics is insufficient to put von Neumann's 1925 proposal in historical context. Von Neumann's 1925 paper can be considered as the first step towards what in modern times goes by the name of NBG set theory. It should to be pointed out that functions are nowadays not emphasized as logical primitives in NBG.
Excerpt from von Neumann's relevant article: Journal für die reine und angewandte Mathematik. Volume 154. Pages 219-240. The yellow passages are what makes this an on-topic contribution to the OP's question. The blue and green passages are to facilitate comparison with the translation
End of translation.
To give a brief summary, partly of things others have already mentioned:
Mathematically, in set theory one can switch between sets and functions rather freely.
Historically, there was once a proposal, in 1925 by von Neumann, to use 'function' as the primitive concept (or, more precisely, to use two primitive sorts13, one called 'argument', the other called 'function', and, interestingly, some but not all 'function's being allowed to be 'argument's, too.
Yet, somewhat ironically, while this proposal indeed was adopted and used, notably by Kurt Gödel in technical mathematical work, it evolved into what nowadays is known as von Neumann–Bernays–Gödel set theory, or 'NBG' for short, and even though von Neumann is part of the eponym, and even though von Neumann argued for using 'function' as a primitive, this was changed by later developers and nowadays in NBG 'function' is not a primitive notion.
There is no clear mathematical reason (known to me) for not making 'function' a primitive notion in NBG.
The 'why' in the OP's title, and some of the OP text, is touching on non-mathematical matters which mathematics intentionally keeps silent about.
Footnotes to the translation.
1 Explanation: in the preceding two pages, which I leave out, von Neumann in charming German had given a history of the first three decades of set theory from Cesare Burali-Forti, arranging the protagonists into two 'groups' according to their, as he puts it, method of 'rehabilitation' of the injured-by-antinomies 19th century set theory; method of rehabilitation of the first group: radical criticisim of the logic used to work with (naive-) sets (recall that set-theory is considered mathematics, not logic, to the point that to this day axioms like those of ZFC tend to be distinguished by terms like 'mathematical axiom' from 'logical axioms', so trying to rescue set-theory by doing something about the logic used to work with sets is indeed usually considered quite a distinct method of 'rehabilitation' than proposing one axiom-system or the other; von Neumann seems to consider this method too extreme [my interpretation], opting himself to use the method of the second group: method of the second group: keep classical logic, yet prohibit the use of naive-sets, rather propose an axiom system in which 'set' appears as a by-itself-meaningless logical primitive, and prove that the system is good. Von Neumann places his work in the second group.
2 I keep von Neumann's choice of words; nowadays this is more-or-less considered a synonym for 'axioms', though not by all, and especially not in the past. I seem to recall that a letter from Hilbert to Frege has survived in which Hilbert feels it necessary to justify why he uses the word 'axiom'. What the fuss is about is that 'axiom' tended to be seen a less neutral term than 'postulate', with 'axiom' having (had) a connotation of 'evidently true by itself', not only a formal 'postulate'. Whether von Neumann chose 'postulate' consciously I do not know, yet it is the more neutral choice of words, more in keeping with his attempt to give a neutral/formal system to do set-theory with (though his choice of title jarrs with this, of course, though this may be purely cultural/linguistic, since 'Eine Postulatisierung der Mengenlehre' would have used a lexically inexistant word and probably wouldn't have found favor with the editors of the journal he is publishing it in.
3 I consciously opt for the unidiomatic literal translation 'free-of-contradictions-proof' instead of the customary yet more opaque term 'consistency proof'.
4 Von Neumann here sounds like he does not really know what to make of Brouwer's 'utterance'.
EDIT October 11, 2017: thanks to F. Ziegler for pointing out that von Neumann here is referencing his own footnote, but the number of this interal footnote was misprinted as 2 instead of 1, for whatever reason.
FORMER VERSION: Please also note that the article of Brouwer's that von Neumann references does not have a page numbered 220., it runs from p. 203 to p. 208 of the relevant journal volume (Jber. Deutsch. Math.-Vereinig. Bd. 28 (1919) S. 203–208); presumably von Neumann references his own page 220 here, in whose footnote I could not find what he means. I cursorily read Brouwer's "Intuitionistische Mengenlehre"; confusingly, there exist at least two articles of Brouwer's with that title, see
and
Upon this cursory reading, I did not see Brouwer writing about the interesting topic of 'intuitionistic consistency proofs for axiom systems' that von Neumann is hinting at here. I simply do not know what he means with his reference to the "Äußerung" of Brouwer's. Maybe this is an interesting question in and of itself, yet I expect this to be very well documented by Brouwer scholars.
5 I consciously use the repetitive-sounding 'construct set-constructions'. This is not meaningless. Constructing the usual set-constructions is an important problem, then and now.
6 Literally, these mean 'axiom-of-filtering-out' and 'axiom-of-replacement'. Roughly, these correspond to what nowadays are called 'Axiom of Separation' and 'Axiom of Replacement'.
7 I consciously avoid translating von Neumann's "Argument-Funktionen" as 'argument-functions', since to contemporary minds this would look misleadingly similar to the $\mathrm{dom}$-function of 1-category-theory, which is something quite different.
8 A perhaps overly charitable reading of von Neumann's original would interpret his use of 'in' in 'in diesen Bereichen' (instead of 'auf diesen Bereichen') as him here eschewing a claim that $[\cdot,\cdot]$ were a total function on a 'collection of all arguments'.
9 Beginning with this formula, von Neumann often omits the comma in the 'operation $[x,y]$. It is not clear whether this is intentional. This is also not addressed in the (short) list of errata which he published soon after. An over-interpretation would be that here von Neumann is coming close to ETCS-style notation $x\circ f$, or $xf$, with $1\xrightarrow[]{x}\mathcal{E}$ a 'global element', and both $x$ and $f$ having the same sort, called 'morphism', this, I think, would be reading to much into this line: this already begins with von Neumann still keeping the 'non-diagrammtical' order of arguments; moreover, while he in this line he indeed temporarily introduces another sort, called 'Variable', and considers both $f$ and $x$ to have this one informal sort, von Neumann here is rather writing on the meta-level: he has written explicitly that formally the first 'variable' is to be considered of sort 'Funktion' and the second 'variable' is to be considered to have the sort 'Argument'.
10 With this 'nur 2 Werte'='only 2 values' von Neumann really means $\leq 2$, since for example to construct the empty set we seem to need a function which takes only one value.
11 Relevant historical background reading on 'Bestimmtheitsaxiom's, roughly from the time of von Neumann's article provides Abraham Robinson:
On the Independence of the Axioms of Definiteness.
The Journal of Symbolic Logic, Vol. 4, No. 2 (Jun., 1939), pp. 69-72.
12 In a strict sense, I think that (0) the accepted answer does not answer the OP's question, (1) this is not the accepted answer's fault since (1.0) the accepted answer does not even claim to answer the OP's question, rather correctly begins with "Let me explain one sense in which using functions or sets provides exactly equivalent foundations of mathematics, in a way that is connected with some deep ideas in set theory. There is a translation back and forth between these foundational choices.", saying what the answer is and (1.1) the OP's question, I think, rests on a premise which I am inclined to deny, since mathematics for the most part, barring perhaps synthetic geometry and some proofs without words, evidently is founded on functions, or function-like ideas, (1.2) the 'why' in the OP's title is a tall order: taken seriously it would entail a frightening tangle of philosophical, psychological, physical, and what not, speculation. There certainly is not the clear-cut mathematical 'answer' for whatever this 'why' is asking a reason for. While it is conceivable that a question of form of the title of the OP might have a rather definite mathematical answer (think of someone asking 'Why aren't Zermelo's natural numbers $\{\}$, $\{\{\}\}$, $\dotsc,$ used predominantly as a model for the (much older, much more international) idea of 'number', instead of von Neumann's ordinals $\{\}$, $\{\{\}\}$, $\{ \{\} , \{\{\}\}\}$, for which there are mathematical reasons. ') , I think that this OP does not have a reasonably precise answer.
13 Perhaps one shouldn't even say 'sort' here. I briefly tried to see BNG in the framework of categorical logic, which is the field which currently uses 'sort' in a precise technical way, and it seems not to fit here. I keep it for pragmatic reasons. Saying 'type' here is not an option, for obvious reasons.