Would mathematics be different if not written one-dimensionally? Mathematics is written one-dimensionally, using symbols that make sense when put together on a line. The 2d sheets of paper that we use don't have enough room to write mathematics two-dimensionally. As a result, most symbols in mathematics are operations involving two objects (or one object acting on another object) and many mathematical concepts seem to depend on this fact (non-commutativity, left/right action).

Question: Would mathematics be different for 4d beings writing on 3d sheets of paper? For example, can it make sense (for them) to extend the notion of left/right action to up/down?

 A: This should be a comment, but I do not know how to put an image there. Some commutative diagrams in homological algebra are in fact depicted "3-dimensionally" (or, at least, using perspective in order to simulate 3-dimensionality).

A: It seems that the main tool of mathematicians is, not a pen or keyboard or other writing tools, but imagination and image-ination. That is, mathematicians think in images, and then maybe in images of images, etc. Of course, images in mathematics are abstract (a bit like those in abstract painting) and concentrated (a bit like those in poetry). Symbols can also be considered images. E.g., when we say "let $X$ denote a separable Banach space", we may think of the concrete 3D or 2D space as an immediate instantiation/source of the symbol $X$. A mathematical formula is a concentrated image of a piece of mathematical text, and it appears that mathematicians of yore did use rather long pieces of text in place of our compact formulas -- which are much easier to grasp, visually and mentally. It further appears that the main original source of image-ination is visual images. Our vision is a very powerful tool. Yet, we can discern, I guess, maybe $10^4\times10^4$ pixels, at most, with our 2D retinas. One might then suppose that 4D mathematicians would be able to discern  $10^4\times10^4\times10^4$ pixels, $10^4$ times as many as we can, with their 3D retinas. Also, their 4D brains would presumably be much, much more powerful than ours. So, they would be able to build much better and more complicated images. Thus, whereas mathematics may be objectively one and the same in our 3D world and their 4D one, one can imagine/image-ine that our most advanced mathematics of today would look to 4D mathematicians as their preschool-level math. 
A: The fact that math is "written" one dimensionally has, I would say, little to do with the information we convey when writing.  It is written linearly because (1) our eyes have a single point of focus, and we can only move that point-of-focus in a 1-dimensional path; and (2) our input processing tends to be synchronous, not asynchronous.  The ideas in mathematics are very very seldom one-dimensional. We have become very good at conveying complex, multi-dimensional ideas with the tools we have.  So no, I don't think it is really significant.
Is there a correlation, i.e. if we were creatures that had multi-dimensional input capabilities, would mathematics be different?  Likely parts would have developed faster or more fully.  Development of good notation, e.g. arabic numerology, has always sped development, but not changed the underlying mathematics, and I would say the same would be true of developing multi-dimensional modes of communication.
But I would say mathematics is mathematics and is independent of the technology of communication.
A: Non-Constructive (Disappointing) Answer: No you don't get anything else by letting your language be two-dimensional. This is because any rigorously-defined 2D language must be a priori defined in the 1D language of normal mathematics/logic. That means you cannot express anything extra in the 2D language -- though you might express the same things in a more natural or insightful manner. For analogous reasons you don't get anything new by using a 3D language.
Example: Double categories (taken from this MathExchange post)

An object in this category is a pair of sets and a map between them. Write $(A \to B)$ for example.
Usually a morphism $f$ in a category has only one range $Dom(f)$ and codomain $Cod(f)$ assigned to it. In a double category there are two flavours of each for every morphism. A morphism in this category is a commutative square like this. . . .
$A \ \to \  B$
$\downarrow  \ \ \ \ \ \ \downarrow$ 
$C \ \to \  D$
The 'vertical' domain and codomain are $Dom_{|} = (A \to B)$ and $Cod_| = (C \to D)$
The 'horizontal' domain and codomain are $Dom_-=(A \to C)$ and $Cod_- =(B \to D)$
There are two 'vertical' and 'horizontal' notions of composition , call them $\circ_|$ and $\circ_-$. But we can only compose two morphisms using $\circ_|$ for example if the corresponding vertical domains match up. What this means is the bottom row of the first matches the top row of the second. 
We compose  vertically by putting the first on top of the second to get a tall rectangle. The corners of the rectangle form a new square. That square is the vertical composition.
Likewise two squares can only be composed horizontally if the right column of the first matches the left column of the second. We compose  horizontally by putting the first left of the second to get a long rectangle. The corners of the rectangle form a new square. That square is the horizontal composition.
It can occur that two morphisms may be composable horizontally but not vertically or vice versa.

Perhaps this satisfies your intuition for what a 2D version of a category should be? But observe we had no trouble defining it in normal 1D mathematics. So while the 2D perspective it might help our intuition it does not add anything in terms of the strength of the language.  
Example: Proof Grids

Suppose $\mathcal L$ is our universe of symbols. In 1D maths a proof is a string of $N$ symbols. That means a map $\{0,1,2, \ldots, N\} \to \mathcal L$. Suppose we want a 2D proof to be a grid  of $M \times M$ symbols (some symbols can be blank). That means a map $\{0,1,2, \ldots, M\}^2 \to \mathcal L$  
But recall the sets $\mathbb N$ and $\mathbb N^2$ have the same cardinality.
  This induces an isomorphism between the classes of 1D and 2D proofs. Thus every 2D proof can already be expressed as a 1D proof.

Example: Digital images

Every (two dimensional) computer image is stored as a (one dimensional) binary sequence.

