There are some kids with that wonderful attitude of asking why?' about anything. The existence of a few syntactically simple axioms beyond which you cannot ask why anymore, as opposed to the discouraging turtles all way down' approach, can be comforting to such minds, I think.
In these cases, I feel that the existence, and even the variety, of foundations should be mentioned, and made intriguing, early, which is a very delicate task.
The problem, of course, is that probably a child isn't generally prepared to face the rigors of formality, and to the complexity of what there is between axioms and doing $2+2$.
This too often leads to never mentioning foundational issues, not even at college: I did physics at university and the most foundational stuff I was ever exposed to were the Cantor set, Dedekind cuts and $\epsilon / \delta$ definitions.
I think that in early education, the operational approach, as opposed to theoretical definitions, can be more appropriate, partly due to the fact that elementary school is expected by the adults to teach children how to perform numerical calculations. Later, however, when the confidence with the object one is manipulating all the time, at least the glimpse of foundations should be given: even only so that those interested can dig it. I sorely regret that didn't happen to me when I was younger, for example.
It is interesting that the opposite approach (heavy set theory from an early stage) was experimented in the past, and with little success: New Math.