If you fix a basis for the irreducible representation $\pi$, then the following elements of the group algebra span the $(\dim\pi)^2$ dimensional space you are interested in.
\begin{equation}
f_{i,j}=\sum_g \pi_{i,j}(g^{-1}) g \,.
\end{equation}
This is a generalization of the fact that characters (or central idempotents) span the group algebra of an abelian group.
So if you find the matrix elements of the two dimensional irreducible representation (the one with two boxes in each of its two rows) of $S_4$, you can plug it into this and obtain the group algebra elements. These matrix elements must have been calculated, but if you want to do it yourself, you could do the following.
Consider the $6$ dimensional combinatorial representation obtained by permuting 4 objects two of which are the same and the other two are the same. A convenient basis is $\{i,j\}$ i.e., choosing two positions out of four. The matrix elements in this basis are not hard to find.
This is actually a representation of $S_3$ as Geoff Robinson pointed out, but we will treat it as a representation of $S_4$ and continue (since as a representation of $S_4$, it is multiplicity free). This $6$ dimensional representation has three irreducible representations of $S_4$: the one dimensional trivial representation, the three dimensional standard representation and the two dimensional one we are interested in. The trivial and the standard span a four dimensional space isomorphic to the defining representation of $S_4$. This defining representation is spanned by the $4$ vectors $\sum_{j\neq 1}\{1,j\}$, $\sum_{j\neq 2}\{2,j\}$, $\sum_{j\neq 3}\{3,j\}$ and $\sum_{j\neq 4}\{4,j\}$. Finding the two dimensional space orthogonal to this is not hard. Then you can find the matrix elements easily and plug it into the above equation to find the group algebra elements.