Here is another, completely different answer.
As Carlo indicated, one can use units in which Planck's constant is $1$. This is no arbitrary choice, but one dictated by fundamental physics. Similarly, one may set the vacuum speed of light to $1$. It is a historical accident (mainly due to our being massive creatures bound to a planet) that we think of length and time as being of different dimensions (in the sense of nonstandard analysis), so that we write $E = m c^2$, but really it's just $E = m$.
Conversely, it's a historical accident (this time a lucky one) that we don't think of force as having its own dimension, so that we have $f = m a$ instead of $f = m a/g$ (where $g$ is Galileo's constant, about $32 \operatorname{lb}_{[M]} \operatorname{ft} \operatorname{s}^{-2} \operatorname{lb}_{[F]}^{-1}$, with a dimension of $[M] [L] [T]^{-2} [F]^{-1}$). This probably only happened because a clear distinction between the pound-force and the pound-mass came after Newton's laws (but before such other units such as the slug, the poundal, the gram, the dyne, and of course the newton).
In electromagnetism, people often make do with only the dimensions $[L]$, $[T]$, and $[M]$, because they set Coulomb's constant to $1$. (Depending on where you put the $c$s, this is either the electrostatic, electromagnetic, or Gaussian system of dimensions; combined with $\operatorname{cm}$, $\operatorname{s}$, and $\operatorname{g}$ as the respective units of $[L]$, $[T]$, and $[M]$, this is called the electrostatic, electromagnetic, or Gaussian system of units.) Only the fuddy-duddies at the BIPM insist on making electric current an independent dimension $[A]$. (They also use $\operatorname{m}$, $\operatorname{s}$, and $\operatorname{kg}$ as the base units, so people write about this as ‘cgs vs mks’, when that's not what it's about at all.)
Similarly, set Boltzmann's constant to $1$ to show that energy and temperature have the same dimension, and set Newton's gravitational constant to $1$ as well. Since of course $1 \operatorname{mol} \approx 6.02 \times 10^{23}$, all $6$ of the physical dimensions implicitly endorsed by the BIPM in the SI system of units (the candela depends on human biology) can now be seen to be utterly dimensionless! (The $6$ constants are Planck's, Maxwell's, Coulomb's, Boltzmann's, Newton's, and Avogadro's, and they are log-linearly independent, giving a unique solution to the system of $6$ homogeneous log-linear equations made by setting them all to $1$.)
The point is: Every quantity is dimensionless, and every unit is simply some real number, so we may calculate with them as if they were real numbers because they are! (In fact, they are all positive real numbers, justifying our use of them in division and inequalities.) The $A^\bullet$ in my first answer is just $A$, and the group in Qiaochu's answer is trivial.
Here is a problem: Although the constants we set to $1$ do come from fundamental physics, there is still some choice (even controversy) about how we do this. First, Planck's constant $h = 2 \pi \hbar$ derives from work on cyclic wave phenomena, and the really basic quantity is Dirac's constant $\hbar = h/(2 \pi)$ (which is also often called Planck's, so Carlo may have meant this all along). Similarly, Coulomb's constant $1/(4 \pi \epsilon_0)$ derives from work on spherically symmetric charge distributions, and the really basic quantity is $1/\epsilon_0$ itself (which, following the Gaussian system on placement of $c$, gives us the Heaviside–Lorentz system of dimensions when we set it to $1$). A similar remark applies to Newton's constant $G = c^2 \kappa/(8 \pi)$; Einstein's constant $\kappa$ is the more basic one. Planck himself, who first came up with all of this, not only used $h$ and $G$ instead of $\hbar$ and $\kappa$, but also used the charge of the proton instead of $\epsilon_0$ or Coulomb's constant, clearly a great error.
So while every unit is a real number, different people disagree over which real numbers they are! (And not just because of experimental uncertainty, which is also an issue somewhat.) All of the possible different conventions to eliminate a given set of dimensions are mediated by a group of symmetries, the group in Qiaochu's answer, so keeping track of them all brings us back to the sophisticated answers that he and I gave.
But the point is: You don't have to choose a convention. Since some convention is possible (and you already knew this when you saw your first system of units, however arbitrary it may have been), it is valid to say that every unit is a real number (even though which real number depends on the convention chosen), and so we may calculate with them as if they were (positive) real numbers.