Let $M$ be an $(R, R)$-bimodule and let $S$ be the square-zero extension $R \oplus M$ of $R$ with multiplication $$(r_1 \oplus m_1)(r_2 \oplus m_2) = r_1 r_2 \oplus (m_1 r_2 + r_1 m_2).$$

If $M$ is free as a left $R$-module then so is $S$, and if $S$ is free as a right $R$-module then $M$ is projective. So it suffices to find $M$ which is free as a left $R$-module but not projective as a right $R$-module.

So let $R = T \times T$ where $T$ is some nonzero ring and let $M = R$ with $R$ acting on the left by left multiplication but acting on the right as follows: the second factor of $T$ acts diagonally by right multiplication and the first factor of $T$ acts trivially. Then $M$ is free as a left $R$-module but torsion as a right $R$-module.

Let $M$ be an $(R, R)$-bimodule and let $S$ be the square-zero extension $R \oplus M$ of $R$ with multiplication $$(r_1 \oplus m_1)(r_2 \oplus m_2) = r_1 r_2 \oplus (m_1 r_2 + r_1 m_2).$$

If $M$ is free as a left $R$-module then so is $S$, and if $S$ is free as a right $R$-module then $M$ is projective. So it suffices to find $M$ which is free as a left $R$-module but not projective as a right $R$-module.

So let $R = T \times T$ where $T$ is some nonzero ring and let $M = R$ with $R$ acting on the left by left multiplication but acting on the right as follows: the second factor of $T$ acts diagonally by right multiplication and the first factor of $T$ acts trivially. Then $M$ is free as a left $R$-module but torsion as a right $R$-module. Taking $T = \mathbb{F}_2$ gives a minimal counterexample in a fairly strong sense; we have $|R| = 4$ and if $|R| < 4$ then $R$ is a field.

Let $M$ be an $(R, R)$-bimodule and let $S$ be the square-zero extension $R \oplus M$ of $R$ with multiplication $$(r_1 \oplus m_1)(r_2 \oplus m_2) = r_1 r_2 \oplus (m_1 r_2 + r_1 m_2).$$

If $M$ is free as a left $R$-module then so is $S$, and if $S$ is free as a right $R$-module then $M$ is projective. So it suffices to find $M$ which is free as a left $R$-module but not projective as a right $R$-module.

So let $R$ be the direct product of countably many copies of the same nonzero ring $T$ and let $M = R$ with $R$ acting on the left by left multiplication but acting on the right by right multiplication after composing with the surjection $R \to R$ which deletes the first factor. Then $M$ is free as a left $R$-module but torsion as a right $R$-module.

Let $M$ be an $(R, R)$-bimodule and let $S$ be the square-zero extension $R \oplus M$ of $R$ with multiplication $$(r_1 \oplus m_1)(r_2 \oplus m_2) = r_1 r_2 \oplus (m_1 r_2 + r_1 m_2).$$

If $M$ is free as a left $R$-module then so is $S$, and if $S$ is free as a right $R$-module then $M$ is projective. So it suffices to find $M$ which is free as a left $R$-module but not projective as a right $R$-module.

So let $R = T \times T$ where $T$ is some nonzero ring and let $M = R$ with $R$ acting on the left by left multiplication but acting on the right as follows: the second factor of $T$ acts diagonally by right multiplication and the first factor of $T$ acts trivially. Then $M$ is free as a left $R$-module but torsion as a right $R$-module.