The group $G:=SL_{\infty}(\mathbb ZQ) = \cup_n SL_n(\mathbb Z)$ Q)$is a concrete example. It is obviously simple and non-amenable. Let$g_n \in SL_{\infty}(\mathbb Z)$Q)$ be the matrix which is $$g_n:= 1_n \oplus \left(\begin{matrix} 0 & 1 \newline -1 & 0 \end{matrix}\right) \oplus 1_{\infty}.$$ and let $$m_{n}(A) := \begin{cases} 1 & g_n \in A \newline 0 & g_n \not \in A \end{cases}.$$ be the finitely additive probability measure associated with $g_n$. Now, for any non-principal ultrafilter $\omega \in \beta \mathbb N \setminus \mathbb N$, $$m(A) := \lim_{n \to \omega} m_n(A) \in [0,1]$$ is a conjugation invariant finitely additive probability measure on $G \setminus {e}$. Conjugation invariance follows since the each element in $G$ commutes with $g_n$ for $n$ large enough.
The group $G:=SL_{\infty}(\mathbb Z) = \cup_n SL_n(\mathbb Z)$ is a concrete example. It is obviously simple and non-amenable. Let $g_n \in SL_{\infty}(\mathbb Z)$ be the matrix which is $$g_n:= 1_n \oplus \left(\begin{matrix} 0 & 1 \newline -1 & 0 \end{matrix}\right) \oplus 1_{\infty}.$$ and let $$m_{n}(A) := \begin{cases} 1 & g_n \in A \newline 0 & g_n \not \in A \end{cases}.$$ be the finitely additive probability measure associated with $g_n$. Now, for any non-principal ultrafilter $\omega \in \beta \mathbb N \setminus \mathbb N$, $$m(A) := \lim_{n \to \omega} m_n(A) \in [0,1]$$ is a conjugation invariant finitely additive probability measure on $G \setminus {e}$. Conjugation invariance follows since the each element in $G$ commutes with $g_n$ for $n$ large enough.