Source for analysis of identification of structures in learner's mind and mathematical structures? Concerning the structure of the learner's mind, psychologist Piaget claimed that 

There exists, as a function of the development of intelligence as a whole, a spontaneous and gradual construction of elementary logico-mathematical structures and that these 'natural' ('natural' the way that one speaks of the 'natural' numbers) structures are much closer to those being used in 'modern' mathematics than to those being used in traditional mathematics. (p. 79 in Piaget 1973). 

Piaget appears to postulate an affinity between, on the one hand, the structures of the mind and, on the other, the structures of modern mathematics (mainly following Bourbaki). The essay in question is 

Piaget, J. "Comments on Mathematical Education," in A. G. Howson, ed., Developments in Mathematical Education: Proceedings of the Second International Conference on Mathematical Education, 79--87, Cambridge: Cambridge University Press, 1973. 

Piaget's postulated affinity has apparently been challenged by some scholars in the context of the New Math controversy. 

Question. Is there a source that provides a detailed analysis of such a postulation of a connection between the structures in the learner's mind on the one hand, and Bourbaki-style structures in the foundations of mathematics, on the other? 

(Note that I am not looking for general sources on the New Math/Modern Math controversy, nor am I particularly interested in Piaget's work in general, but rather for an analysis of this particular identification of Piaget's psychological structures and Bourbaki's mathematical structures).
I just came across a book that might be relevant: 

All Positive Action Starts with Criticism: Hans Freudenthal and the Didactics of Mathematics. By Sacha la Bastide-van Gemert. Springer, 16 Jan 2015 

Here the author quotes Freudenthal as follows on page 211: 

It thus did not begin with the Sputnik shock. It had already begun in the early 1950s. They had even managed to convince Piaget, who did not understand anything of it except for the fact that the word "structure" appealed to him. With Piaget's name on the billboard they felt confident of the support of psychology. What now, psychology! Mathematics is ruled by a logical order and he who teaches mathematics is easily seduced to sacrifice the psychological, the educational order to the logical order. I have done my utmost to avoid this and in my 'fragment Rekendidactiek' of 1942, if not earlier, I wanted to warn others. But what was now happening before my very eyes? A logical order brought to ecstasy, a systematic of mathematics as a whole--that is how mathematics should be taught. 

It is clear from this that Freudenthal was sceptical of these developments but unfortunately he does not elaborate the details of his objections. 
 A: You say that Freudenthal does not elaborate his objections, but in a sense he did just that in his book Mathematics as an educational task (1973), Appendix I, starting: “The somewhat summary criticism I administered on several occasions
to Piaget's work demands more detailed argument.” However, this addresses egregious mathematical misconceptions and methodological flaws (ill-chosen questions) more than the (“ontogeny parallels phylogeny”?) postulation you ask about. 
Against that, his argument seems much shorter:

(p. 46): Bourbaki. How convincing this organization of mathematics is! So
  convincing that Piaget could rediscover Bourbaki's system in 
  developmental psychology. Poor Piaget! He did not fare much better than Kant,
  who had barely consecrated Euclidean space as "a pure intuition" when
  non-Euclidean geometry was discovered! (...) Mathematics is
  never finished – anyone who worships a certain system of mathematics
  should take heed of this advice.
(p. 192): Impressed by Cantor's analysis he turned to studying the development of the number concept under this aspect. (...) Piaget believed that the concept of natural number could be entirely derived from potencies. (...) This may have seduced him to believe that it is
  also psychologically true; it was one of his ideas to trace in 
  developmental psychology the system of mathematics he happened to be 
  acquainted with.

Or, as reported by F. Goffree in The legacy of Hans Freudenthal (1993):

(p. 36): Piaget thought that the cognitive development in children took place from poor to rich structures whereas HF thought it was the other way round. Geometry showed something similar. In the Erlanger Program Klein had given a hierarchy of geometrical structures: topological, projective, affine and euclidean, in HF's terms from poor to rich. Children start by drawing irregular circles, anyone can see that. This was sufficient reason for Piaget to presume that their geometric development began
  with the topological structure. HF remarked condescendingly that those very same children were quite capable of distinguishing between correctly drawn circles and other figures.

(Note added: These last remarks, not sourced by Goffree, are in Revisiting mathematics education (1991), p. 27. So far as I can tell, the criticized “postulation” is most detailed in Épistémologie des mathématiques. Partie II. Chap. 8 & 11 (1961, transl. 1966).)
A: A much-cited attempt to analyze Piaget's ideas and carry them further has been given by Ed Dubinsky in Reflective abstraction in advanced mathematical thinking (1991). 

Reflective abstraction is a concept introduced by Piaget to describe
  the construction of logico–mathematical structures by an individual
  during the course of cognitive development. Two impor- tant
  observations that Piaget made are first that reflective abstraction
  has no absolute beginning but is present at the very earliest ages in
  the coordination of sensori–motor structures and second, that it
  continues on up through higher mathematics to the extent that the
  entire history of the development of mathematics from antiquity to the
  present day may be considered as an example of the process of
  reflective abstraction.
In the majority of his own work, however, Piaget concentrated on the
  development of mathematical knowledge at the early ages, rarely
  going beyond adolescence. What we feel is exciting is that, as he
  suggested, this same approach can be extended to more advanced topics
  going into undergraduate mathematics and beyond. It seems that it is
  possible not only to discuss and conjecture, but to provide evidence
  suggesting, that concepts such as mathematical induction,
  propositional and predicate calculus, functions as processes and
  objects, linear independence, topological spaces, duality of vector
  spaces, duality of topological vector spaces, and even category theory
  can be analyzed in terms of extensions of the same notions that
  Piaget used to describe children’s construction of concepts such as
  arithmetic, proportion, and simple measurement.

A: Daniel Everett described in his book 
Dont sleep there are snakes the language of an Amazonian tribe (Piraha) which has no words for numbers. This tribe also uses "immediacy of experience" for believing other people. The author also argues that this language is a counter-example to the idea of a language instinct as proposed by Whorf and Chomsky.
Caleb Everett in Numbers and the Making of Us: Counting and the Course of Human Cultures argues, based on the language of the Piraha, that counting is a cultural achievement and not innately laid down in the structure of the brain. 
On page 274, in "Notes" to the section "6. Quantities in the Minds of Young Children", Caleb E. mentions Piaget as follows:


*

*

*Jacques Mehler and  Thomas Bever, “Cognitive Capacity of Very Young Children,” Science 3797 (1967): 141–142. See also the enlightening discussion on this topic in Dehaene,  The Number Sense: How the Mind Creates Mathematics, particularly as it relates to the work of Piaget. I should mention, however, that an insightful reviewer notes that there have been issues replicating the results of Mehler and Bever with very young children.



A review of Caleb Everett's book is in the New York Times. 
Addition:
In the second part of his book, Caleb E. summarizes empirical evidence that people with anumerate languages and human babies have an innate sense to recognize quantities up to 3 precisely, and another innate sense to recognize the larger quantity between two aproximately (like: 16 is more than 8). To connect the two and to start to count needs exposure to language with number words and is a slow process. The approximate number sense is also there in rats, etc. 
On page 168 he writes:


*

*As Elizabeth Brannon and Joonkoo Park, two animal cognition specialists, recently suggested: “it is challenging to understand how such a primitive system that is not capable of representing exact large numbers could give rise to the formal mathematics that is uniquely human.”20  The quantitative thought with which we and other species are innately equipped is orders of magnitude removed from the kinds of quantitative thought that most humans eventually possess.  This suggests that biological explanations of such thought are inherently limited. Most of our numerical cognition owes relatively little to our neurobiological equipment and owes much more to the ways we manipulate that equipment.  This manipulation can only be possible if there are external tools interacting with our innate mechanisms for quantity differentiation.  The principal external tools in question are numbers, symbolic representations of quantities that are linguistically reified and used in culturally variant ways.  The existence of numbers explains the gulf between actual human numerical thought and the numerical thought to which we are innately predisposed.

A: Grothendieck wrote a text entitled "Structure de la psyché" that might help shed a light on the issues you consider. As a former member of Bourbaki, it seems highly plausible that he tried doing so to describe how the brain works mathematically, perhaps from a categorical point of view. I don't know whether this text is available online, but imho there's no doubt it should be quite instructive. Maybe the book by Philippe Douroux contains some information on this, but I haven't read it yet.
