Let $p$ be a prime number. Is there a noncommutative reduced ring of order $p^2$? (Note that any ring of order $p^2$ with identity is commutative).
Let $A$ be a reduced ring of order $p^n$. Then $(pa)^n = p^n a^n = 0$ for all $a \in A$ so $pA = 0$ and $A$ is an algebra over $\mathbb{F}_p$.
Adjoining an identity element to $A$ if necessary, we may view $A$ as an ideal (of codimension at most $1$) in a finite dimensional unital $\mathbb{F}_p$algebra $B$. Let $J$ be the radical of $B$ and $C = B/J$.
Since $A$ is reduced and $J$ is nil, $A \cap J = 0$ so $A$ is isomorphic to an ideal in $C$. Write the finite semisimple ring $C$ as a direct sum of simple rings $C = S_1\oplus \cdots \oplus S_m$; then $A \cap S_i$ is either $S_i$ or $0$ for all $i$. Assume without loss of generality that $S_i \subseteq A$ for $i=1,\ldots, k$ and $A \cap S_i = 0$ for $i > k$. Then $A$ contains $S_1 \oplus \cdots \oplus S_k$.
We can find unique central idempotents $e_i \in S_i$ such that $S_i = e_iC$ and $1 = e_1 + \cdots + e_m$. Because $A$ is an ideal in $C$, if $a \in A$ and $i > k$ then $e_i a \in A \cap e_i C = A \cap S_i = 0$ so $a = e_1a + \cdots + e_ka \in S_1 \oplus \cdots \oplus S_k$.
Hence $A = S_1 \oplus \cdots \oplus S_k$ is a finite direct sum of finite simple rings. By ArtinWeddernburn, each $S_i$ is a matrix ring over a finite division ring. These division rings are commutative by another theorem of Wedderburn, and the matrix sizes involved must all be $1 \times 1$ because otherwise $A$ would not be reduced. Therefore $A$ is commutative, and actually is isomorphic to a finite direct product of finite fields.
Note that if the reduced assumption on $A$ is dropped, then any nonzero onesided ideal in the matrix ring $M_2(\mathbb{F}_p)$ gives an example of an associative, nonunital, noncommutative ring of order $p^2$.

$\begingroup$ I don't understand the Tensor part is your solution and after that when you concluded that $A$ has an ideal complement in $C$. Would you please explain more ? $\endgroup$– user37834Sep 11 '13 at 11:18

$\begingroup$ Sure. I've tried to give more details above without using tensor products. $\endgroup$ Sep 11 '13 at 14:58
A more conceptual path to essentially the same solution:
Lam, a first course in noncommutative rings, pag. 207: (12.7) Theorem. A nonzero ring $R$ is reduced iff $R$ is a subdirect product of domains. [Yes, it works also for unitless rings]
Finite subdirect product of rings are direct products: Chinese remainder theorem. [This is an optional step, not needed to obtain only commutativity but used to obtain the full structure of the ring]
Finite domains are division rings (since injective endofunctions of a finite set are surjective)
Now apply Wedderburn (finite division rings are Galois fields) [Kaplansky, Jacobson, Herstein et. al. have many generalizations of Wedderburn's theorem (whose proof, however, uses Wedderburn's result, plus steps related to the first stem in this proof)].
With the same proof (without Wedderburn):
reduced (right) artinian rings are exactly the finite direct products of (possibly skew) fields.