A remark of Connes on non-standard analysis In an interview (at http://www.alainconnes.org/docs/Inteng.pdf) Connes remarks that

I had been working on non-standard analysis, but after a while I had found a catch in the theory.... The point is that as soon as you have a non-standard number, you get a non-measurable set. And in Choquet’s circle, having well studied the Polish school, we knew that every set you can name is measurable; so it seemed utterly doomed to failure to try to use non-standard analysis to do physics.  

What does he mean; what is he referring to?
 A: A Google search turns up a page on nonstandard analysis on WorldLingo that cites a quote from Connes in the section on criticisms.  This may give you more understanding of what Connes is referring to, given that this seems to be a re-statement of what you have quoted above:

The answer given by nonstandard analysis, a so-called nonstandard real, is equally deceiving. From every nonstandard real number one can construct canonically a subset of the interval [0, 1], which is not Lebesgue measurable. No such set can be exhibited (Stern, 1985). This implies that not a single nonstandard real number can actually be exhibited.

The next remark is:

A. Connes Noncommutative Geometry and Space-Time, Page 55 in The Geometric Universe, Huggett et al. The point of Connes' criticism is that nonstandard hyperreals are as fictitious as non-measurable sets. These sets can be shown to exist, assuming the axiom of choice of set theory, but are not constructible. Non-measurable sets are usually considered pathological, a sort of irritant that must be tolerated in order to have the axiom of choice available.

As there are sources listed, you may be able to get some additional insight in reading some of the references.
http://www.worldlingo.com/ma/enwiki/en/Non-standard_analysis#Criticisms
A: 
...as soon as you have a non-standard number, you get a non-measurable set.

Every nonstandard natural number $N$ gives rise to a nonprincipal ultrafilter $U$ on $\mathbb{N}$, by saying that a set $X\subset\mathbb{N}$ is in $U$ if and only if $N\in X^*$, the nonstandard analogue of $X$. In other words, the ultrafilter says $X$ is large if it expresses a property that the nonstandard number $N$ has. We may regard $U$ as a subset of $2^\mathbb{N}$, which carries a natural probability measure. But a nonprincipal ultrafilter cannot be measurable there, since the full bit-flipping operation, which is measure-preserving, carries $U$ exactly to its complement, so $U$ would have to have measure $\frac12$, but $U$ is invariant by the operation of flipping any finite number of bits, and so must have measure $0$ or $1$ by Kolmogorov's zero-one law. (See also this article by Blackwell and Diaconis proving the same fact.)

...every set you can name is measurable.

Another way of saying that a set is easily described is to say that it lies low in the descriptive set-theoretic hierarchy, and the lowest such sets are necessarily measurable. For example, every set in the Borel hierarchy is measurable, and the Borel context is often described as the domain of explicit mathematics. Under stronger set-theoretic axioms, such as large cardinals or PD, the phenomenon rises to higher levels of complexity, for under these hypotheses it follows even that all sets in the projective hierarchy are Lebesgue measurable. This would include any set that you can define by quantifying over the reals and the integers and using any of the basic mathematical operations.
A: A recent article by Leichtnam and myself (arxiv) in the American Mathematical Monthly contains a "theorem" to the effect that, in the presence of a construction of the hyperreals, the following is true: as soon as you have a Connes infinitesimal, you get a non-measurable set.
A: I don't think this answer is fundamentally different from Joel's, but perhaps the differing exposition may help.
Every irrational real number in $[0,1]$ has a unique binary expansion, and so every irrational real number in ${}^\ast[0,1]$ has a unique $\ast$-binary expansion. The set of irrational real numbers whose $N$-th binary digit is 0 (where $N$ is an infinite nonstandard natural number) is not Lebesgue measurable. This follows from the Lebesgue Differentiation Theorem.
I'm slowly beginning to grasp that adjectives like "countable" and "measurable", although applied to individual sets, actually describe the ambient set theory and not the set itself.

Edit: The weakest form of the Lebesgue Differentiation Theorem is this: Let $A\subseteq[0,1]$ be measurable. For almost all $x\in A$,
  $$ \lim_{r\to0^+} \frac{\lambda(A \cap (x-r,x+r))}{2r} = 1.$$
Such $x$ are often called "points of density". More involved statements allow one to consider more general measure spaces than $[0,1]$, to integrate functions (instead of taking the measure of a set), and most interestingly, to consider more general types of neighborhoods than balls of radius $r$. For "nice" neighborhoods this generalizes, and there is still some issue as to whether particular neighborhoods are too ugly to be nice, or not.
The take-away is this: a measurable set has its positive measure in clumps. The set described above is too uniform to be measurable.
A: Specifically with regard to the issue of what Connes means when he says that you can't "name" an infinitesimal: this issue was discussed above on this page, and I think we provided an answer that's different from what was given above. Namely, Connes specifically refers to Solovay. Recall that in Solovay's model, all measurable sets are definable. This indicates that what Connes means by "naming" is that infinitesimals are not definable. From this Connes jumps to the conclusion that the hyperreal theory is "virtual". This, we argue, is a non-sequitur, or even an error. A hyperreal field is indeed definable, as was shown by Kanovei and Shelah to everybody's surprise. Therefore Connes' claim is erroneous. As far as the fact that infinitesimals are not definable, well Connes himself used ultrafilters in an essential manner in his earlier work in functional analysis (including the papers mentioned by Choi above). In this sense Connes is criticizing his own earlier work, in a way. Note that a generic real number is arithmetically not definable, so the whole thing is a non-starter. 
An additional point is that Skolem's non-standard model of the integers imbeds in *R. A non-standard number in Skolem's model is represented by a definable function on N, and therefore represents a Robinson hypernatural. Skolem's nonstandard integers can be constructed without the axiom of choice. Yet they can also be viewed as Robinson non-standard integers. This makes it particularly clear that the fact that a non-standard integer produces a non-measurable set is due not the "chimerical" nature of the integer (as Connes repeatedly claimed) but rather to the power of the transfer principle (which is available in Robinson's framework but not in Skolem's).
A: Connes' critique was recently analyzed by Kanovei, Katz, and Mormann in this article in Foundations of Science (see also arXiv 1211.0244). Here is the abstract:

We examine some of Connes' criticisms of Robinson's infinitesimals starting in 1995. Connes sought to exploit the Solovay model S as ammunition against non-standard analysis, but the model tends to boomerang, undercutting Connes' own earlier work in functional analysis. Connes described the hyperreals as both a 'virtual theory' and a 'chimera', yet acknowledged that his argument relies on the transfer principle. We analyze Connes 'dart-throwing' thought experiment, but reach an opposite conclusion. In S, all definable sets of reals are Lebesgue measurable, suggesting that Connes views a theory as being 'virtual' if it is not definable in a suitable model of ZFC. If so, Connes' claim that a theory of the hyperreals is 'virtual' is refuted by the existence of a definable model of the hyperreal field due to Kanovei and Shelah. Free ultrafilters aren't definable, yet Connes exploited such ultrafilters both in his own earlier work on the classification of factors in the 1970s and 80s, and in Noncommutative Geometry, raising the question whether the latter may not be vulnerable to Connes' criticism of virtuality.
We analyze the philosophical underpinnings of Connes' argument based on Goedel's incompleteness theorem, and detect an apparent circularity in Connes' logic. We document the reliance on non-constructive foundational material, and specifically on the Dixmier trace (featured on the front cover of Connes' magnum opus) and the Hahn-Banach theorem, in Connes' own framework. We also note an inaccuracy in Machover's critique of infinitesimal-based pedagogy.

A brief review of Kanovei-Shelah is here.
Connes wrote in 2001 as follows:

What conclusion can one draw about nonstandard analysis? This means that, since noone will ever be able to name a nonstandard number, the theory remains virtual (Connes et al. 2001, p. 16)

The exact meaning of the verb "to name" used by Connes here is not entirely clear. Connes provided a hint as to its meaning in 2000, in the following terms:

if you are given a non standard number you can canonically produce a subset of the interval which is not Lebesgue measurable. Now we know from logic (from results of Paul Cohen and Solovay) that it will forever be impossible to produce explicitely [sic] a subset of the real numbers, of the interval [0, 1], say, that is not Lebesgue measurable (Connes 2000a, p. 21, 2004, p. 14).

The reference to Solovay indicates that Connes is relying on the result, which may be found in Solovay (1970, p. 3, Theorem 2) on the existence of a model S of set theory ZFC, in which (it is true that) every set of reals definable from a countable sequence of ordinals is Lebesgue measurable. Thus when Connes claims that the theory remains virtual he apparently means that the theory remains undefinable.
This however is contradicted by the existence of a hyperreal field definable in ZF, with transfer probable using only the countable axiom of choice, and a further mild well-orderability assumption (weaker than well-orderability of the reals taken for granted by Connes) ensuring properness; see this answer for details.
