If $\lambda\in\overline{\mathbb{Q}}$, the elliptic curve $$ E_\lambda\colon y^2=x(x-1)(x-\lambda) $$ has $(\lambda,0)$ as $2$-torsion point and is defined over (a subfield of) $L=\mathbb{Q}(\lambda)$. Its Weil restriction $A_\lambda:=\operatorname{Res}_{L/\mathbb{Q}}(E_\lambda)$ is an abelian variety defined over $\mathbb{Q}$ and shares the same points of $E_\lambda$, including their torsion structure, so $\lambda\in \mathbb{Q}(A_\lambda[2])$.
EDIT As Urlich observed, what I wrote was wrong/useless: indeed, the defining property of Weil restricition is that the $\mathbb{Q}$-points of $\operatorname{Res}_{L/\mathbb{Q}}$ coincide with the $L$-ones of $E_\lambda$, so the point corresponding to $(\lambda,0)$ is defined over $\mathbb{Q}$. Instead of $\operatorname{Res}_{L/\mathbb{Q}}$, (assume $L/\mathbb{Q}$ Galois, which makes no harm, and) consider $A_\lambda=\prod_{\sigma\in G(L/\mathbb{Q})}E^\sigma$, which is an abelian variety over $\mathbb{Q}$ and has the required property: the point $$ \big((\lambda,0),O,O,\dots,O\big) $$ where $O$ is the unit has the required property (namely, it generates $L$ and is torsion): thanks to Felipe for the solution.