# Suggestions for good notation

I occasionally come across a new piece of notation so good that it makes life easier by giving a better way to look at something. Some examples:

• Iverson introduced the notation [X] to mean 1 if X is true and 0 otherwise; so for example Σ1≤n<x [n prime] is the number of primes less than x, and the unmemorable and confusing Kronecker delta function δn becomes [n=0]. (A similar convention is used in the C programming language.)

• The function taking x to x sin(x) can be denoted by x ↦ x sin(x). This has the same meaning as the lambda calculus notation λx.x sin(x) but seems easier to understand and use, and is less confusing than the usual convention of just writing x sin(x), which is ambiguous: it could also stand for a number.

• I find calculations with Homs and ⊗ easier to follow if I write Hom(A,B) as A→B. Similarly writing BA for the set of functions from A to B is really confusing, and I find it much easier to write this set as A→B.

• Conway's notation for orbifolds almost trivializes the classification of wallpaper groups.

Has anyone come across any more similar examples of good notation that should be better known? (Excluding standard well known examples such as commutative diagrams, Hindu-Arabic numerals, etc.)

• In set theory we write ${}^B A$ for the set of functions from $B$ to $A$. – Andrés E. Caicedo Oct 20 '10 at 20:54
• I've always assumed that the notation $A^B$ is because of the "exponential law" $(A^B)^C = A^{B\times C}$ ... – Kevin H. Lin Oct 20 '10 at 23:18
• Yes, among other things. Also $A^B\times A^C=A^{B+C}$, where $+$ is disjoint union. But all the great reasons for it don't help for our mind thinking that maps start with the source and end with the image, not the other way round. – darij grinberg Oct 20 '10 at 23:20
• Arabic numerals ? Ah yes, they were transmitted to Europe by the Arabs. – Chandan Singh Dalawat Oct 21 '10 at 3:29
• Isn't $x \mapsto f(x)$ commonplace? As for homomorphisms, they are not simply maps, and $\mathrm{Hom}(A, B)$ denotes the whole class, while $A \to B$ denotes a single mapping. – Alexei Averchenko Oct 21 '10 at 3:38

$f_*$ and $f^*$ for direct and inverse image. We really should use this right from the beginning, for functions $f\colon X\to Y$, where $f_*\colon P(X)\to P(Y)$ ($P(X)$ being the power set) and $f^*\colon P(Y) \to P(X)$ instead of the awful notations $f(A)$ and $f^{-1}(B)$ for subsets $A$ of $X$ and $B$ of $Y$.

• Why do you think $f(A)$ and $f^{-1}(B)$ are awful? – Anton Petrunin Nov 9 '10 at 0:32
• I would say they are extremely confusing for beginners. Sometimes they think there is an inverse map $f^{-1}$. – Leo Alonso Nov 9 '10 at 15:40
• Per Leo, it is indeed confusing for beginners, and not surprisingly so: f is being used both for a function from X to Y and for a function from P(X) to P(Y). This can even be ambiguous as well as confusing e.g., when the domain is a set such as {0,{0}}. And I further agree that the notation $f^{-1}$ for inverse image is likely to suggest there is an inverse function. It is but a small consolation that when there is an inverse function, the notations are self-consistent (because $f^*=(f^{-1})_*$ in that case). – David MJC Nov 9 '10 at 20:36
• I have also seen $f^{\rightarrow}$ and $f^{\leftarrow}$. – Max Nov 14 '10 at 11:39
• I like $f^\rightarrow$ and $f^\leftarrow$ better, since when applying the inverse, it looks like: $(f^{-1})^\rightarrow = f^\leftarrow$. – Hsien-Chih Chang 張顯之 Dec 18 '10 at 15:25

Dirac's bra-ket notation

This notation is very useful when applying the Hilbert spaces in Quantum Theory. It exploits some properties of duality, eigenvalues/eigenvectors, projectors and self-adjoint operators. In mathematics, perhaps it is difficult to adopt, because mathematicians are using notations that are more general, and cannot exploit these particularities. But if you know Hilbert spaces you can learn this notation in one minute, and then it makes visible many of these nice properties.

• I have to strongly disagree with the use many physicists do of the Dirac bra-ket notation, as I'm going to explain: mathematicians would use the $\langle\, |\, \rangle$ as a function with the "blank places" to be saturated by its 2 argument (or even $\langle\, | \, | \, \rangle$ as a function to be saturated by 3 arguments, the one in the middle being an operator as in $\langle \xi | \, A | \,\eta \rangle$) [continued] – Qfwfq Oct 22 '10 at 13:33
• [continued] On the contrary, I have seen many physicists use the highly "dis-functional" notation $| \, \lambda \rangle$ to denote an eigenvector with $\lambda$ as eigenvalue, and so write things like: $A | \lambda \rangle=\lambda\cdot | \lambda \rangle$ – Qfwfq Oct 22 '10 at 13:37
• There are other examples of notations which don't represent the missing arguments by blanks. One is the (abstract) index notation. An index which appears repeated in covariant and contravariant positions means contraction, and if it is not repeated, it represents a free slot that can be filled by contracting with another free slot of another tensor. The blank to be filled by an argument is not visible, but we know that a free index can do this. Another example is the Polish notation, where again we don't see the missing arguments as blanks. – Cristi Stoica Oct 23 '10 at 9:46
• @unknown (google). On the other hand, I like $|\lambda\rangle$ as eigenvector only if it is stated that $\lambda$ has multiplicity 1, otherwise is confusing. When multiplicity is 1, the confusion is avoided because in fact $|\lambda\rangle$ is a ray in the complex projective space. – Cristi Stoica Oct 23 '10 at 10:11
• There was a long, long discussion at the n-Category Cafe one time, started by how offensive "bra" was to someone (speaking as a woman). – Todd Trimble Oct 15 '12 at 19:18

Here is a notation in algebraic geometry that in my opinion is very useful and self-explanatory but not used widely.

For a birational morphism $f:X\to Y$ there exists an open dense set $U\subseteq Y$ for which $f$ induces an isomorphism $f^{-1}U\to U$. For a closed subset $Z\subseteq Y$ such that $Z\cap U\neq\emptyset$ the strict transform is defined as $$\overline{f^{-1}(Z\cap U)}\subseteq X,$$ i.e., the closure of the preimage of the part of $Z$ that lies on the part where the morphism is an isomorphism. This is a very important construction and there isn't a universally accepted notation for it.

János Kollár invented the following notation for this: $$f^{-1}_*Z:= \overline{f^{-1}(Z\cap U)}\subseteq X$$ The genius of the notation is that anyone familiar with basic notation in algebraic geometry should understand what it is:

1) As $f$ is birational, $f^{-1}: Y\dashrightarrow X$ exists as a rational map.

2) For any map $g$, it is common to use $g_*$ to denote push-forward of cycles.

The strict transform is really just the push-forward of cycles via the rational map $f^{-1}$.

Diagrammatic notation for tensors (Penrose diagrams, birdtracks, etc.). It makes many things like the invariance of tr(A B C) under cyclic permutation into empty statements.

The ever-controversial reverse Polish notation for functions: $f(x) = xf$. Thus in composition, the order makes sense: $(g \circ f)(x) = x f g$ (this point is moot for the fortunate Hebrew- and Arabic-speaking mathematicians). I hate this notation in practice but I can't deny that it is objectively right and "just makes sense" in more or less the same way that the original post discusses writing $B^A = A \to B$. Please no one vote this up.

• One partial compromise is to subscript a function $f: X \to Y$ as $f_{Y \leftarrow X}$. For instance, the composition of two inclusion maps $\iota_{Z \leftarrow Y} \circ \iota_{Y \leftarrow X}$ becomes $\iota_{Z \leftarrow X}$. – Terry Tao Oct 21 '10 at 17:11
• Hehe, objectively right'. – Greg Muller Oct 21 '10 at 18:42
• @Terry Tao: I've seen that one from time to time, and it is pretty nice when you are dealing with different spaces (especially when you have a category of them and, as you write, a functorial assignment of maps). For self-maps of X, it leaves...something to be desired. – Ryan Reich Oct 21 '10 at 20:36
• Abstractly an arrow from $X$ to $Y$ has and an orientation but no direction. It could be drawn either up/down/right/left/slant 30°/slant-45°/... . But we are heavy victims of the typographical habits, and this in spite of modern computer possibilities (not means :Tex being a terrible tool!). The best thing to do is to think of an arrow as slanted +200°or in three dimensions, then the typographical induction tends to disappear completely (at least for me and I guess for Hebrew/Arabic writers too). So the motto (not necessarily a categorist's one) is "let's do multidimensional algebra!". – Jérôme JEAN-CHARLES Nov 23 '10 at 23:50
• Haskell uses & for reverse application and >>> for reverse composition. I've also seen ;` used for reverse composition. – user76284 Oct 27 '19 at 0:02

The universal property of the univariate polynomial ring: For any commutative ring $A$, any commutative $A$-algebra $B$ and any $x\in B$, there exists one and only one $A$-algebra homomorphism from the polynomial ring $A\left[X\right]$ to $B$ which maps $X$ to $x$.

This is the so-called evaluation homomorphism at $x$. I denote this homomorphism by $\lim\limits_{X\to x}$. This has the advantage that we have $\lim\limits_{X\to 0}\dfrac{\left(X+1\right)^n-1}{X}=n$ and similar properties hold just as in classical analysis. The polynomial $\dfrac{\left(X+1\right)^n-1}{X}$ is well-defined (since $X$ is not a zero divisor in $A\left[X\right]$ and divides $\left(X+1\right)^n-1$), but if we would blindly replace $X$ by $0$ we would obtain a $\dfrac{0}{0}$ error.

• Ah, I really like calculus-inspired notation for algebraic stuff, just like the integral for ends. Looking forward to see more in this spirit! – Konrad Voelkel Dec 19 '11 at 10:54
• This looks beautifully created :) very neat – Mr Pie Jan 27 '18 at 22:07

For quite some time, Giovanni Sambin has been advocating the notation $A\between B$ for "overlapping" sets, that is, for inhabited intersections, where $\exists_{x} \ x \in A \cap B$. He makes his case in a constructivist frame, but I think that a lot of ink, chalk---and keyboard pushing...---would be spared if one wrote $A\between B$ instead of $A\cap B \neq \emptyset$, even in non-constructive mathematics.

• Doesn't extend nicely to $A\cap B\cap C\ne\varnothing$. – Akiva Weinberger Sep 1 '15 at 23:57
• Well, one can just overload the overlapping symbol, $\between (A, B, C)$; it would again be more economical than the traditional exploitation of the associativity of $\cap$. – Basil Sep 2 '15 at 7:37
• So we could write $\between (A, B) := (A\cap B = \varnothing)$ as well? – Mr Pie Jan 27 '18 at 22:08
• One must be careful with this type of notation, because one can be confused as to whether $A\between B$ denotes a set, or a statement about $A$ and $B$. A fix is to say: "we write $A\between B=C$ when we have $A \cap B=C$ and $A\cap B\neq \varnothing$", and to refrain from writing things like "now consider the set $A \between B$". – nombre May 14 '19 at 3:58
• Nice. I'd use it not for referring to a set (consider the set $A \between B$) but a statement (consider $A$ and $B$ such that $A \between B$. – Arnaud Chéritat Dec 25 '19 at 13:49

I like $A^{\text{H}}$ for the conjugate transpose of the matrix $A$, ananlogously to how $A^{\text{T}}$ and $A^{\text{C}}$ means the transpose and the conjugate. You call it the Hermitian of the matrix for short. I learnt this notation from Rózsa Pál, but I can't tell who invented it.

• I use simply $A^\ast$ for this but I also saw $A^\dagger$ for both transpose and conjugate. – Marcel Bischoff Dec 18 '10 at 13:10
• I also like $A^{-H}$ for $(A^H)^{-1}=(A^{-1})^H$. Often handy, but I suggest you not to try $A^{2H}$ or $A^{\frac12 H}$ (exercise: why is the latter ill-defined?) – Federico Poloni Dec 19 '11 at 10:49

A) Two notations I love are the rising factorial $x^\overline n$ and its falling factorial twin $x^\underline n$. They are used and advocated in the great book see http://en.wikipedia.org/wiki/Concrete_Mathematics . In passing this book uses great notations.

B) A general trick with binomials to reuse them with sets instead of numbers, here are some typical examples.

1) $\binom S k$ to denote the set of all $k$-sets of the base set $S$ .

2) $S^\underline 2$ to denote the pairs $(x,y)$ of $S$ where $x$ and $y$ are different.

3) $S^\underline k$ to denote the $k$- uplets of $S$ (each uplet has $k$ different elements).

C) Another notation I find useful when listing some (big) families of examples in a combinatorial setting. Use as variables the very numerals $1$ $2$ .. themselves instead of $x_1$ , $x_2$ ... . For example ( very untelling because too small an example) : the intersection of $123$ and $34$ is $3$.

D) I also often use {{ a,a,b,c}} for multiset. Any other standard or suggestion (or a way to avoid speaking about multiset) is welcome.

• D) Perhaps {$(a)^2, b, c$} for your multiset {{a,a,b,c}} – Qfwfq Sep 24 '11 at 21:04
• for unknowngoogle : this is even better without the parenthesis around a , yet it does not work with variables ${{a1, a2}}$ where you don't know if $a1$ equals $a2$. – Jérôme JEAN-CHARLES Sep 28 '11 at 11:54
• Perhaps $2a+b+c$ for your multiset: a 0-cycle. – Ben McKay Dec 1 '13 at 10:12
• @BenMcKay, I agree, since a (finitary) multiset on the symbols $X$ is just an element of the free Abelian monoid generated by $X$, which thereby becomes a "module over the semiring $\mathbb{N}$" in a canonical way. – goblin GONE May 6 '14 at 7:37
• (C) reminds me of a pet peeve of mine, which is unnecessarily indexed sets. I frequently see something like "$\prod\limits_{i \in I} x_i$, where $\{x_i : i \in I\}$ is the set of primes not exceeding 200" rather than "$\prod\limits_{\substack{\text{$x$prime} \\ x \le 200}} x$" (and other less transparently silly examples). – LSpice Jul 7 '19 at 12:29

In commutative algebra with many variables, repeating lists of variables in polynomial arguments and various rings gets very tedious. I suggest using $X_{1..n}$ instead of $X_1,\ldots,X_n$.

Here's an excerpt from Bourbaki's Commutative Algebra, page 222:

For every formal power series $f\in A[[X_1,\ldots,X_n]]$, $$f(X_1,\ldots,X_n)-f(Y_1,\ldots,Y_n)=\sum_{i=1}^n (X_i-Y_i)h_i(X_1,\ldots,X_n,Y_1,\ldots,Y_n)$$ where the $h_i$ belong to $A[[X_1,\ldots,X_n,Y_1,\ldots,Y_n]]$.

And here's how it looks with my suggested notation:

For every formal power series $f \in A[[X_{1..n}]]$, $$f(X_{1..n})-f(Y_{1..n})=\sum_{i=1}^n (x_i-Y_i)h_i(X_{1..n},Y_{1..n})$$ where the $h_i$ belong to $A[[X_{1..n},Y_{1..n}]]$.

I think this notation is maximally succinct, and helps a reader from getting lost in long lists of variables.

• I can see that being handy when one needs to call out some of the indices. For your example from Bourbaki, I would sooner use X decorated with a hat or overbar to indicate a tuple. Gerhard "That May Just Be Me" Paseman, 2013.02.03 – Gerhard Paseman Feb 3 '13 at 8:26
• You could combine with mathoverflow.net/a/42941 to get the still more succinct $X_{\underline n}$. – LSpice Jul 7 '19 at 12:35

I would love for topologists to start differentiating between properties that hold “locally” and properties that hold “regionally”, that is: A local property holds on some neighbourhood basis of every point, a regional property holds on some neighbourhood of every point. For any space, having a property locally implies having it regionally.

This would resolve the permanent confusion of “which sort of local” is meant in a given context.

• There's something mathematically funny about a comment that begins "I would love for topologists to start differentiating". 😄 – LSpice May 19 at 13:01

This is probably the exact opposite of what the thread starter intended, but here's an instance where it might have been useful to have overloaded notation! I recently found that in propositional logic, $p \to q$ obeys much the same rules as exponentiation $q^p$: for instance, we have $(r^q)^p = r^{q \times p}$, and similarly, $p \to (q \to r)$ iff $p \land q \to r$. This is apparently due to the universal property for exponential objects, as applied to a Boolean algebra viewed as a poset category. I suppose it's also an instance of the Curry—Howard correspondence.

More generally, it seems like it isn't such a bad idea to conflate exponentiation and arrows - it looks nicer, to me at least, to write that a function of the type $A \to (B \to C)$ is naturally isomorphic to a function of the type $A \times B \to C$, than to write about $A \to C^B$. Even, as some have suggested, $A \to {}^B C$ or $C^B \leftarrow A$ would look nicer. On the other hand we'd lose the association with cardinal arithmetic if we do this...

If $\mathcal{C}$ is a category and $X,Y\in\mathrm{obj}(\mathcal{C})$, I like the notation $\mathcal{C}(X,Y)$ to denote $\mathrm{Hom}_{\mathcal{C}}(X,Y)$.

So, $\mathcal{C}(X,X)=\mathrm{End}_{\mathcal{C}}(X)$.

What do you think of the notation $\mathcal{C}(X):=\mathrm{Aut}_{\mathcal{C}}(X)$ ?

This would be consistent with the notation (or similar notations) $\mathsf{DIFF}(S^1)$ (resp. $\mathsf{TOP}(S^1)$ ) for diffeomorphisms (resp. homeomorphisms) of the circle, i.e. the $\mathrm{Aut}$ in the category $\mathsf{DIFF}$ of smooth manifolds (resp. $\mathsf{TOP}$ of topological manifolds), sometimes used in topology (see e.g. here and here. And (see e.g. here) $\mathsf{TOP}(n)=\mathrm{Aut}_{\mathsf{TOP}}(\mathbb{R}^n)$.

• Is there any particular reason you've chosen to make your links single letters, instead of pointing them out more explicitly? – S. Carnahan Dec 10 '12 at 3:25
• @S. Carnahan: not really... I just wanted the links to be kind of parenthetic references, because those MO pages weren't "standard" definitory places for the notations. If it's considered weird of annoying according to MO netiquette, I'll just edit :) – Qfwfq Dec 11 '12 at 14:09
• I'm using these notations privately already, especially the $\mathcal{C}(X)$ one and I didn't know there was anyone else doing it! I'd upvote twice if I could! – Manuel Bärenz Mar 25 '14 at 16:44
• I have been using some similar notation for a while too! Just like you do, I wright $\mathcal{C}(X,Y)$ for morphisms, but for monomorphisms I write a "monomorphism arrow" over $\mathcal{C}$, and similarmly for epimorphisms and isomorphisms. For endomorphisms I abbreviate $\mathcal{C}(X)$, and for automorphisms I write $\mathcal{C}(X)$ with a double-sided arrow above the C. – Pedro G. Mattos Jan 17 at 5:59

If one needs to denote the fiber (not the stalk which is standardly denoted $\mathcal{F}_{x}$) of a sheaf $\mathcal{F}$ at the closed point $x$ of the $\Bbbk$-scheme $X$, one can write

$\mathcal{F}\mid_{x}$

After all, the fiber $\mathcal{F}\otimes_{\Bbbk}\;\kappa (x)$ is the restriction (pullback) of $\mathcal{F}$ to the point $x:\rm{Spec}\;\Bbbk\rightarrow X$.

The problem is that, when you identify vector bundles with locally free sheaves, the above notation clatches with the usual notation $E_x$ for the fiber of vector bundles. On the other hand almost always the context would be sufficient to clarify which of the two notations is being used.

• In Ravi Vakil's thread on teaching schemes, there was some discussion of using the notation $\mathcal{F}(x)$, since given a section $f$ of the sheaf, its values as a function on $X$ are written $f(x)$ and lie in the fibers. – Ryan Reich Oct 21 '10 at 6:38
• It's this answer, and the many, many comments below it: mathoverflow.net/questions/28496/…. Note that what I wrote is given by BCnrd about halfway down. – Ryan Reich Oct 21 '10 at 6:48
• @Ryan: I appreciate the $\mathcal{F}(x)$ notation, but there's a (minor?) clutching with the widely used notation $\mathcal{O}_{X}(x)$ to denote the line bundle on the algebraic curve $X$ twisted by the divisor given by the point $x\in X$. – Qfwfq Oct 21 '10 at 11:44
• It's true, this notation would lead to the unfortunate equality $\mathcal{O}_X(x) = \mathcal{O}_X/\mathcal{O}_X(-x)$. – Ryan Reich Oct 21 '10 at 16:22
• I also like the notation $x \prec y$ to denote majorization of a vector $x$ by a vector $y$; once defined, this notation relieves quite lot of burden.

• On a related note, I also prefer the notation $A \succeq 0$ to signify that $A$ is a positive semidefinite matrix (some prefer to use the perhaps "more natural" $A \ge 0$, but since I frequently deal with nonnegative matrices, the $\ge$ is out)

• Unfortunately, "majorization" has at least two different meanings... – darij grinberg Oct 22 '10 at 10:45
• Majorization theory involves also the useful notation $x_\uparrow$ and $x_\downarrow$ for vectors with real entries. – Denis Serre May 18 '11 at 15:14

All of the notations created to simplify writing category theory. For instance, the idea of drawing a circular arrow inside of a diagram to indicate that that diagram is commutative. As well as the idea of putting an angle in the top left or bottom right of a square diagram to indicate that it is a pushout or pullback. And finally, the notation of augmenting any of these notations with $\simeq$ to indicate that the diagram is only "up to homotopy".

I find $$\lim_{x\nearrow0} f(x)$$ $$\lim_{x\searrow0} f(x)$$ for the limit from below and from above much more intuitive that all other notation I've seen, including $$\lim_{x\to 0^-}f(x)$$ and $$\lim_{x\to 0^+}f(x)$$.

• For $\lim_{x\to0^+}$ or $\lim_{x\searrow0}$ one could almost imagine $\lim_{0\leftarrow x}$... but then $\lim_{x \to0}$ would become ambiguous. – Zach Teitler May 13 '19 at 19:14
• I have seen $\lim\limits_{x\downarrow 0}$ and $\lim\limits_{x\uparrow 0}$ – Mr Pie May 13 '19 at 21:32
• @ZachTeitler, obviously the solution is to write $\lim_{x \rightarrow 0 \leftarrow x}$ for the two-sided limit. :-) – LSpice Jul 7 '19 at 12:40
• Obviously, not $\lim_{0 \leftarrow x \rightarrow 0}$, that would be ridiculous... :-) – Zach Teitler Jul 8 '19 at 23:13

The three-dot notation $f\mathrel{\scriptsize\vdots}A\to B$ to indicate that $f$ is a partial function from $A$ to $B$, meaning that $\text{dom}(f)\subseteq A$ rather than $\text{dom}(f)=A$. Partial functions are pervasive in logic, especially computability theory and set theory, and this notation is both compact and suggestive.

• I use $\rightharpoonup$ for partial functions. – Andreas Blass Oct 20 '10 at 22:23
• Use a broken arrow instead: f : A - - - > B. I have no idea how three dots is suggestive of the domain being something smaller than A. – KConrad Oct 21 '10 at 18:15
• I just meant that it suggests that $f$ is something like a function from A to B, without being intrusive. This notation is really useful in situations where you have numerous partial functions of different arities running around. – Joel David Hamkins Oct 21 '10 at 18:22
• Never saw the three-dot notation. It looks like a smudge or tiny dead gnat to me. I use the notation mentioned by Andreas. – Todd Trimble Oct 30 '10 at 18:28
• I prefer Andreas's notation but have also seen $f :\subseteq A \rightarrow B$ which can be read mnemonically as "f is a function from a subset of A to B" or "f is a subset of a function from A to B" (this not being strictly true, unless we identify a function with its graph). – Max Nov 14 '10 at 11:36

Forgive me for bumping such an old big-list question, but I can't resist. In linear algebra texts you will sometimes find the notation $$[v]_B$$ for the vector of coordinates of a vector $$v \in V$$ with respect to a basis $$B$$ of $$V$$, but you will not (to my knowledge) find corresponding notation for the matrix of a linear transformation $$T : V \to W$$ with respect to two bases $$B, C$$ of $$V$$ and $$W$$, except perhaps in the special case $$V = W, B = C$$, where I have seen $$[T]_B$$. The notation I use and advocate for this matrix is

$$_C[T]_B$$

and I taught this in a few linear algebra classes at UC Berkeley as a TA although I don't think it caught on. It has the pleasant property that its definition can be written

$$_C [T]_B [v]_B = [T(v)]_C.$$

This lets us give a transparent proof of the equally pleasant "functoriality" property that if $$S : U \to V$$ is another linear transformation and $$A$$ is a basis of $$U$$ then

$$_C[TS]_A = (_C[T]_B)(_B[S]_A)$$

("the $$B$$s just cancel"). Try even stating this result without notation for it! The notation makes it hard to misstate the basic facts about how such matrices change upon change of basis etc., because the correct statements "typecheck" properly and you won't be tempted to multiply matrices that haven't been expressed in compatible bases. Although things would be slightly better if either 1) the vector notation were written with the basis on the left or 2) matrices acted on the right and we switched the order of the two bases.

Importantly, the change-of-basis matrix between two bases $$B, B'$$ can just be written $$_B[I]_{B'}$$ and all of its properties become obvious (everything follows from "canceling" bases) and easy to remember, including e.g. that $$_{B'}[I]_B$$ is the inverse matrix and that if $$T : V \to V$$ is an endomorphism then

$$_{B'}[T]_{B'} = (_{B'} I_B) (_B[T]_B)(_B I_{B'}).$$

$$\LaTeX$$ balks a bit at the repeated subscripts, though.

(Incidentally, it's annoying that we use "vector" to denote both an "abstract vector" and a "concrete vector" while we have different terms for "linear transformation" and "matrix." Perhaps we should start using something like "array" or "list" or "tuple" to mean a concrete vector.)

• Related: Notation for change of basis matrix, where I suggested we adopt the partial derivatives notation for this. – Michael Bächtold Sep 4 '20 at 7:59
• Perhaps it would be better to write $_{C^{-1}}[T]_B$ so that $_B$ and $_C$ could be interpreted as isomorphisms between $V$ and $W$ and $\mathbb R^n$ and $\mathbb R^m$? – Oscar Cunningham Sep 4 '20 at 8:17
• Row vectors should be $[v]_B$, column vectors should be ${}_B[v]$ (or $_{B^{-1}}[v]$ perhaps). – Zach Teitler Sep 4 '20 at 15:53
• At the Charles university in Prague a lot of people teach with similar notation. Except they take it one notch further and write it as $[T]^C_B.$ This has the added advantage of exposing students to kind of "Einstein summation notation" common in physics and differential geometry. – Vít Tuček Sep 8 '20 at 19:57
• Another advantage is that it allows you to make a clear distinction between vectors $[v]^B$ and forms $[\alpha]_C$ and between linear mappings $[T]^C_B$ and bilinear forms $[T]_{C,B}.$ – Vít Tuček Sep 8 '20 at 19:58

To say that $u$ and $v$ are orthogonal you can spell out "The scalar product of $u$ and $v$ is equal to zero", i.e.:

$\langle u,v \rangle=0$

but you can also use the binary symbol $\perp$ to write the sentence "$u$ orthogonal to $v$" more directly, i.e. $u\perp v$.

Analogously, to say that sets $A$ and $B$ have empty intersection, of course you can spell out "$A$ intersection $B$ equals the empty set", i.e.:

$A \cap B = \emptyset$

But it would be nice if there was a binary symbol (like a barred $\cap$ symbol, not to be confused with the $\pitchfork$ symbol for transversality) to say directly "$A$ does not intersect $B$ (nontrivially)".

I don't think this symbol already exists in LaTeX.

• You could use $\perp$ for that as well... – Harry Altman Oct 22 '10 at 13:28
• I write $A\thinspace )\thinspace(\thinspace B$ for $A\cap B=\emptyset$ and similarly $A\supset\hskip-5pt\subset B$ for $A\cap B\ne\emptyset$. – Christian Blatter Nov 9 '10 at 9:26
• Surely you should write $A \supset \subset B$ and not $A)(B$. – Ryan Reich Nov 16 '10 at 10:43
• You can use $A \parallel B$ for $A\cap B=\varnothing$, since parallel lines don't meet, and so do disjoint sets. – Asaf Karagila Oct 13 '11 at 17:31
• @Asaf that metaphor for || doesn't work if A and B are subsets of projective space... SCNR. – Konrad Voelkel Dec 19 '11 at 10:59

I really like $(-)^n$ instead of $(-1)^n$ for alternating signs in series etc. Its more aesthetic and slightly easier to write. I found it in a 1976 paper about multipolar expansions.

• I have used this in my own personal notes. I've deliberately avoided showing it to many of my pre-calc or calculus students though. They have enough issues with notation as is. – JoshuaZ May 21 '20 at 11:13

(This would be a comment on notation for partial functions, but I don't have the reputation points, as I just joined MO.) Though this is by no means standard, for personal use I've adopted the following system of arrow decorations that captures many standard types of binary relations. For a relation f from A to B, use $\rightharpoonup$ to indicate $\forall x\in A ~\exists y \in B~~xfy$, $\rightharpoondown$ to indicate $\forall x\in A~\exists^{\leq 1} y\in B~~xfy$, $\leftharpoondown$ to indicate $\forall y\in B~\exists x\in A~~xfy$, and $\leftharpoonup$ to indicate $\forall y\in B~\exists^{\leq 1}x\in A~~xfy$. So, $\rightarrow$ is for functions, $\leftrightarrow$ is for bijections, $\leftharpoonup\hspace{-1em}\to$ is for injections, $\leftharpoondown\hspace{-1em}\to$ is for surjections, $\rightharpoondown$ is for partial functions, $\rightharpoonup$ is for serial relations, and so on.

• I was sold when I saw the notation for injections, but I'm not so happy with the notation for surjections, as compared to the usual symbol $\twoheadrightarrow$. There is nothing in that arrow to suggest "surjection" to me. – Ryan Reich Nov 16 '10 at 10:46
• Incidentally, this answer also indirectly introduces the notation $\exists^{\le1}$, which has the marvellous virtue that its meaning is so obvious no-one wondered about, or even visibly noticed, it. – LSpice Jul 7 '19 at 12:25

Has this already been mentioned? If a group $G$ acts on a commutative group $A$ by homomorphisms, $G \to Aut(A)$, then use $a^g$ to denote the action. Especially if the group multiplication on $A$ is written multiplicatively, where we can say things like $(ab)^g = a^g b^g$. This can come up especially in Galois theory; I remember Lang using this notation in his Algebra to prove Hilbert's Theorem 90, and I thought it was very neat, and enhanced the readability of notation as well.

• I like this notation for actions in general, but for the particular case of actions on groups written in multiplicative notation, this is quite confusing, as superscripts already denote too many other things in this context. You ruled out conjugation by making the group commutative, but it’s still ambiguous which superscripts in a thing like $(a^kb)^g$ denote action by $G$, and which denote powers with integer exponents. – Emil Jeřábek Dec 7 '13 at 13:09
• @ToddTrimble, I agree with Serre (Cohomologie Galoisienne, p. 42): "Si $s \in G$ et $x \in E$, le transformé $s(x)$ de $x$ par $s$ sera souvent noté ${}^sx$ [mais jamais $x^s$, pour éviter l'horrible formule $x^{(st)} = (x^t)^s$]." (Nonetheless, Harish Chandra does use your suggestion for the conjugation of a group on itself or its Lie algebra.) – LSpice May 18 '15 at 19:36
• I use a similar notation for any group action on any set (or structured set); By putting the exponent on the left, one can also distinguish between left- and right-actions. For example the two possibilities for conjugation would be ${^g x} = gxg^{-1}$ and $x^g = g^{-1}xg$. I learned this notation from Burkhard Külshammer. – Johannes Hahn Sep 4 '20 at 8:44

Whenever $X$ is a singleton set, denote by $!X$ the unique element in $X$, or put else (assuming some standard material set theory), $$! \colon \mathrm{Singletons} → \mathrm{Set}$$ is the unique inverse for $\{\,\} \colon \mathrm{Set} → \mathrm{Singletons},~x ↦ \{x\}$. So $!\{x\} = x$ for all things $x$ and $X = \{!X\}$ for all singletons $X$.

This makes it possible to formulaically talk about “the unique element $x$ such that …” (which happens a lot) and it fits well with the notation “$∃!$” for saying “there exists some unique …”.

For example, one may define the minimal polynomial of an element $α$ in an algebraic field extension $E / F$ as $$\operatorname{minpol}_F α = ~!\{f ∈ F[X];~\text{f monic, irreducible with f(α) = 0}\}.$$

• By the way, I use angle brackets $⟨~⟩$ whenever the curly braces of sets feel awkward, e.g. in the context of defining maps, I’d rather write $$\mathrm{minpol}_F \colon E → F[X],~α ↦ ~!⟨f;~\text{f monic, irreducible with f(α) = 0}⟩.$$ – k.stm Jul 15 '17 at 4:01

Round brackets as Cartesian coordinates and square brackets as homogenous coordinates.

I picked this up idea from Needham's Visual Complex Analysis, and I'm not sure how commonly it's used elsewhere. In the book, the convention was to use round-bracketed matrices $$\begin{pmatrix} a & b \\ c & d \\ \end{pmatrix}$$ for representing linear transformations, and square-bracketed matrices $$\begin{bmatrix} a & b \\ c & d \\ \end{bmatrix}$$ for representing Möbius transformations. More generally, we can let round brackets be for all "ordinary", "Cartesian", "vector"-ish things, and square brackets be for all "projective" or "homogeneous" things. It's useful to be able to tell them apart quickly, as the algebra looks the same but the meaning is very different.

Outside of complex analysis, there's an even plainer context where this convention would help: computer graphics (a.k.a. the actual "real-life" application of projective geometry). In computer graphics, there are two common ways to think of a point, say, in 2-dimensional space. One way is a pair $(x,y)$, the way people normally think of coordinates. The other way is to add an extra coordinate: $x$ and $y$ along with an extra $1$. This is a practical representation, since it works naturally with affine and projective transformations, which we represent in homogeneous coordinates too. The use of both systems leads to a problem. Without any notational convention, is $(3,-4,1)$ supposed to mean a 2D point or a 3D point? Is $$\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \\ \end{bmatrix} \begin{bmatrix} -2 \\ 3 \\ 1 \\ \end{bmatrix}$$ supposed to be a translation applied to a 2D point, or a shear mapping applied to a 3D point? You need context and you can't tell at a glance. If we used round/square brackets with our designated interpretations, though, we'd know right away that $(3,-4,1)$ means a 3D point and the square-bracketed matrix above means a 2D translation.

Unfortunately, they don't actually make any notational distinction at all in computer graphics. They're just going to stay confused ;) - but you don't have to. You can use the round and square to distinguish ordinary and projective coordinates. It deserves to be more standard.

(There's another convention out there to make the distinction with commas and colons, i.e. $(a,b,c)$ is Cartesian and $(a \mathbin{:} b \mathbin{:} c)$ is homogeneous. It works well for some purposes, but not if you need the notation to generalize to matrices.)

For a Lie group $$G$$ with Lie algebra $$\mathfrak{g}$$, and element $$A\in\mathfrak{g}$$, denote the left invariant vector field $$\overrightarrow{A}(g)=L_{g*} A$$, and the right invariant vector field $$\overleftarrow{A}(g)=R_{g*}A$$. On any matrix group, $$\overrightarrow{A}(g)=gA$$ and $$\overleftarrow{A}(g)=Ag$$, easy to remember. The notation uses fewer subscripts than the more common notation $$X_A$$, which is also not easy to adapt to right versus left invariant vector fields. When discussing group actions, we usually use left actions. The left action of the group on itself is generated by the right invariant vector fields. Some authors use $$X_A$$ for left invariant vector fields, but also for the generators of any action of the group, which can lead to sign mistakes when discussing the left action of the group on itself.

• The visual asymmetry between the lengths and heights of the arrows $\overleftarrow A$ and $\overrightarrow A$ is a bit unfortunate …. – LSpice May 19 at 12:59

The lack of a nice obviously symmetric notation for $\binom{a+b}{b}$ has bothered me; Dijkstra suggested in EWD 782 the notation $P(a,b)$, generalizing it also to $P(a_1,\ldots,a_k)$ for $\binom{a_1+\ldots+a_k}{a_1,\ldots,a_k}$. (Though I certainly disagree with him about $\binom{n}{k}$ being useless - you certainly do want to think about it that way a lot of the time.) I haven't actually had any reason to use this since I saw it but I can certainly think of times I would have.

Also the double-parentheses multichoose notation $\left(\!\binom{n}{k}\!\right)$ is nice because it lets you say "...and this is n multichoose k (which is equal to this binomial coefficient)" instead of just jumping directly to a binomial coefficient whose relevance may not be immediately obvious. But I suppose that's not really on the level of giving you a better way to look at things.

• Isn't there a nice obviously symmetric notation for ${a+b} \choose b$, namely ${a+b} \choose {a,b}$? – Rasmus Oct 21 '10 at 18:34
• Oh, true. That is a good point. It looks a bit clunky but it works. – Harry Altman Oct 21 '10 at 20:28
• $\left(\!\binom{n}{k}\!\right)$ – JBL Oct 22 '10 at 13:26
• is given by \left(\!\binom{n}{k}\!\right) – JBL Oct 22 '10 at 13:27
• $\binom{a+b}{a,b}$ often means something else when $a=b$, namely the number of ways of partitioning $2a$ into two groups of size $a$. Thus $\binom{4}{2,2}$ is $3$ not $6$. – David MJC Nov 10 '10 at 19:40

Multi-factorials are handy. Sometimes results can be expressed compactly by introducing a double factorial or possibly higher factorial. For example

$$\int_0^{\pi/2} \sin^{2n+1} \theta \:\: d\theta = \frac{(2n)!! }{ (2n+1)!!}$$

• I wish there were a notation that didn't scream "iterated factorial", though (not that one sees this very much). I forget: does $n?$ mean anything? The question mark is handy because it suggests having to make a choice, as in "even or odd?" – Ryan Reich Nov 16 '10 at 10:48
• The question mark has a meaning in C and programming languages derived from C. The notation a ? b : c; means to do b if a is true, otherwise do c. The question mark also means "optional" in regular expressions. For example, the regular expression ab?c matches abc or ac. I don't know whether either of these notations would make sense imported into math. On a related note, sometimes I would like to import C's % operator into math notation. – John D. Cook Nov 16 '10 at 23:30
• True, but C also doesn't have a factorial operator, and ! means something entirely different again. There's not much reason to make mathematical notation agree with programming design choices. As for %, we always have "mod". – Ryan Reich Nov 29 '10 at 9:08
• The problem with "mod" is that it is usually an equivalence relation and not a function. That is, you see "a equiv b mod m" more than "a mod m". I'm not sure the latter is common notation or that people agree in detail what it means. – John D. Cook Nov 29 '10 at 15:22

I think that Inuit numerals are cool. (http://en.wikipedia.org/wiki/Inuit_numerals) They are useful for vigesimal type things.

• Now I wonder where in mathematics you would like to do actual vigesimal calculations? – Konrad Voelkel Dec 19 '11 at 10:44

Using $(a, b, ... )$ is handy to denote a column vector, which is the transpose of the row vector $[a, b, ... ]$, especially in linear text. Correspondingly, all displayed matrices should be written with brackets, not parentheses. This notation agrees with the usual identification of coordinates with column vectors.

• I take it you also hate having to write the transpose when you're constrained to writing on one line? :) – J. M. isn't a mathematician Oct 21 '10 at 11:45
• Yes, particularly when stacking vectors $a, b, ...$, in which case you have to write the artificial $[a^T, b^T, ... ]^T$ without this convention. – John Bentin Oct 21 '10 at 13:52
• It is, of course, not $[a, b, \dots]^T$ but $[a\;b\;\dots]^T$. This is an awful convention because even once you remind yourself why the $T$ is there you (or I at least) am not convinced that it has any meaning other than to satisfy a badly chosen precedent. I don't know why anyone would vote this down. – Ryan Reich Oct 21 '10 at 16:27
• Don't forget the invariant literature which uses $\left[v_1,...,v_n\right]$ not for the matrix formed by the columns $v_1$, ..., $v_n$, but for its determiannt... – darij grinberg Oct 21 '10 at 17:04
• I personally prefer the notation $(a_1,\ldots,a_n)$, with commas, or equivalently $(a_1\;a_2\;\ldots\;a_n)^{\mathrm{T}}$ or $[a_1\;a_2\;\ldots\;a_n]^{\mathrm{T}}$ for column vectors. And $(a_1\;a_2\;\ldots\;a_n)$ or $[a_1\;a_2\;\ldots\;a_n]$ for row vectors. – Qfwfq Jan 19 at 0:00