show/hide this revision's text 3 Added a possible answer to the original question (in the update)

Update

Having thought about the question some more, I think that perhaps it is meant in the following sense. Quantization, whatever it is, should be a process by which one goes from a classical hamiltonian system to a quantum mechanical system. The former is given by a Poisson manifold $P$ and a choice of hamiltonian, the latter by a Hilbert space $\mathcal{H}$ and a self-adjoint operator. Quantization should relate classical states and classical observables to quantum states and quantum observables. In particular, it should relate points in $P$ with rays in $\mathcal{H}$. Hence one possible hope would be that quantization be a functor from the category of Poisson manifolds to the category of (projectivised?) Hilbert spaces. Perhaps it is in this sense that the phrase in your question is meant. I was trying to remember where it was that I read that for the first time, but so far nothing. It was a long time ago...

show/hide this revision's text 2 Put the reference in the right place!

I believe that the phrase "quantization is not a functor" originated in the days before deformation quantization. Originally, based perhaps on the very few examples of quantum systems that people had at their disposal, the hope was that one could quantize a classical hamiltonian system by the simple procedure of replacing (up to $i\hbar$) the Poisson bracket of functions by the commutator of operators. In other words, suppose that $P$ is a classical phase space and $H \in C^\infty(P)$ a hamiltonian function, with time evolution given by the hamiltonian vector field $\lbrace H,-\rbrace$. Quantization would then be a map $f \mapsto O_f$ from $C^\infty(P)$ (the classical observables) to self-adjoint operators in a Hilbert space in such a way that $$[O_f,O_g] = i\hbar O_{\lbrace f,g\rbrace}.$$

A discussion of this is given in Section 5.4 of Abraham and Marsden's Foundations of Mechanics.

It was soon realised that this could not work for all classical observables. The first result of this kind is the so-called Grönewald/Van Hove's theorem, which shows that ordering ambiguities force the above equation to be true only up to terms of higher order in $\hbar$.

A discussion of this is given in Section 5.4 of Abraham and Marsden's Foundations of Mechanics.

In general, it is accepted that one has to make choices in quantization. Even in the context of deformation quantization, the deformation may not be unique. Moyal quantization, for example, corresponds to a choice of ordering prescription (Weyl's symmetric ordering).

There are also subtler effects. For example, it is possible to cook up one-dimensional quantum mechanical systems for which the hamiltonian is not self-adjoint, but admits inequivalent self-adjoint extensions, each one giving rise to different physical predictions.

I realise that this does not actually answer your question, which as far as I understand it asks between which categories does quantization fail to be a functor. My point is that quantization may not even be a map!

show/hide this revision's text 1

I believe that the phrase "quantization is not a functor" originated in the days before deformation quantization. Originally, based perhaps on the very few examples of quantum systems that people had at their disposal, the hope was that one could quantize a classical hamiltonian system by the simple procedure of replacing (up to $i\hbar$) the Poisson bracket of functions by the commutator of operators. In other words, suppose that $P$ is a classical phase space and $H \in C^\infty(P)$ a hamiltonian function, with time evolution given by the hamiltonian vector field $\lbrace H,-\rbrace$. Quantization would then be a map $f \mapsto O_f$ from $C^\infty(P)$ (the classical observables) to self-adjoint operators in a Hilbert space in such a way that $$[O_f,O_g] = i\hbar O_{\lbrace f,g\rbrace}.$$

A discussion of this is given in Section 5.4 of Abraham and Marsden's Foundations of Mechanics.

It was soon realised that this could not work for all classical observables. The first result of this kind is the so-called Grönewald/Van Hove's theorem, which shows that ordering ambiguities force the above equation to be true only up to terms of higher order in $\hbar$.

In general, it is accepted that one has to make choices in quantization. Even in the context of deformation quantization, the deformation may not be unique. Moyal quantization, for example, corresponds to a choice of ordering prescription (Weyl's symmetric ordering).

There are also subtler effects. For example, it is possible to cook up one-dimensional quantum mechanical systems for which the hamiltonian is not self-adjoint, but admits inequivalent self-adjoint extensions, each one giving rise to different physical predictions.

I realise that this does not actually answer your question, which as far as I understand it asks between which categories does quantization fail to be a functor. My point is that quantization may not even be a map!