Let $a=-1$ and pick $b$ be an element such that $\sigma_1(b),\sigma_2(b)$ are positive, while $\sigma(b)$ is negative for all other $\sigma:F\to\mathbb R$. Such elements exist by suitable approximation theorems in number fields. For $A$ defined using these elements, after tensoring by $\mathbb R$ using the first element we get a matrix algebra, while using the other $\sigma$'s we recover Hamilton's quaternions.

It remains to guarantee $A$ itself is not split. If $[F:\mathbb Q]>2$ this is clear as there is some non-split infinite place. Otherwise, we can pick $b$ to satisfy a suitable approximation at any finite place, so as to guarantee $A$ still has some non-split place.

Let $a$ be any totally negative element and pick $b$ to be an element such that $\sigma_1(b),\sigma_2(b)$ are positive, while $\sigma(b)$ is negative for all other $\sigma:F\to\mathbb R$. Such elements exist by suitable approximation theorems in number fields. For $A$ defined using these elements, after tensoring by $\mathbb R$ using the first element we get a matrix algebra, while using the other $\sigma$'s we recover Hamilton's quaternions.

It remains to guarantee $A$ itself is not split. If $[F:\mathbb Q]>2$ this is clear as there is some non-split infinite place. Otherwise, we can pick $a,b$ to satisfy suitable approximations at any finite place, so as to guarantee $A$ still has some non-split place.