An algebraic number is not a root of unity? This problem is related to my study of the Burau representation of the braid group $B_3$: I was trying to show that certain "congruence subgroups" are of infinite index.
There is an approach that boils it down to the following question:

Let $\xi$ be a primitive root of unity of degree $2n>12$ (just in case). Then the roots $\lambda$ of $\chi(\lambda):=\lambda^2+(\xi^2-\xi+1)\lambda+\xi^2$ are not roots of unity.

I was particularly interested in the cases $n=7$ or $9$. In both cases, $\deg\Phi_{2n}=6$ (here $\Phi$ is the cyclotomic polynomial) and, hence, $\mathbb{Q}[\lambda]$ has degree at most $12$. There are finitely many (although quite a few) cyclotomic polynomials $\Phi_k$ with $\deg\Phi_k\le12$. For each such polynomial, one computes the $\lambda$-resultant $R(\xi)$ of $\Phi_k$ and $\chi$ and checks that $R(\xi)\ne0\bmod\Phi_{2n}(\xi)$.
Clearly, this approach should work for any given $n$, but I have no idea how this can be done "in general". So, here is the question:

Is there a smarter way to prove that an algebraic number is not a root of unity?

 A: Assume that the roots $\zeta$ and $\xi^2/\zeta$ of your equation are roots of unity. Then by Vieta Theorem $\zeta+\xi^2/\zeta=-(\xi^2-\xi+1)$; equivalently sum of 5 roots of unity $\zeta+\xi^2/\zeta+\xi^2-\xi+1$ vanishes. All such relations are classified in Theorem 6 of paper by Conway and Jones. This implies the result.
A: You are interested in the solutions of the bivariate equation $V\colon f(X,Y)=X^2+X(Y^2-X+1)+Y^2=0$ in roots of unity, a sort of question known as "Manin-Mumford problem". The answer is known in the greater generality of subvarieties of algebraic groups — except for finitely many points, they are explained by the existence of "torsion subvarieties".
In the case you're interested in, the answer has been proved by Ihara, Serre, Tate, see Lang, Division points on curves. 
The proof is quite elementary. Let $z=(\xi,\eta)$ be a point of $V$ where $\xi,\eta$ are roots of unity; let $n$ be the lcm of their orders. Then, for every integer $d\geq2$ which is prime to $n$, there exists $\sigma\in\mathop{\rm Gal}(\overline{\mathbf Q}/\mathbf Q)$ such that $\sigma(z)=z^d$. Then, $z$ is
a common root of $f(X,Y)$ and of $f(X^d,Y^d)$. By Bézout's theorem, the locus defined by these two equations is either equal to $V$ (if $f(X,Y)$ divides $f(X^d,Y^d)$), or is finite, of cardinality $\leq \deg(f)^2 d=9d$. In the first case, $f$ (which is irreducible) would be the difference of two monomials,
which it isn't. So we are in the latter case. 
Since all conjugates of $z$ belong to this finite set, one 
gets the inequality $\phi(n)\leq 9d$.
By the prime number theorem, it is possible to choose $d$ quite small with respect to $n$, say $d\ll \log(n)$. (The worst case is when $n=p_1\dots p_r$, the product of the $r$ first primes—then, $d=p_{r+1}$ is possible, and (PNT), one has $p_{r+1}\sim r\log(r)$ while $\log(n)=\sum_{i\leq r} \log(p_i)\sim r\log(r)$ as well, so that $p_{r+1} \approx \log(n)$.) Similarly, $\phi(n)\gg n^{1-\epsilon}$, hence $n^{1-\epsilon}\ll 9 \log(n)$. This bounds $n$ from above, and it remains to check every possible $z$ in a finite set.
A: I assume that $n=p$ be a prime and $p>3$: indeed, if $p=3$ then your statement is false because $\xi^2-\xi+1=0$. Your polynomial is
$$
\chi(\lambda)=\lambda^2+(\xi-1)^2\lambda+\lambda\xi+\xi^2\in\mathbb{Z}[\xi]
$$
where $\xi$ is a primitive $2p$-th root of unity. Let $\zeta$ be a root of $\chi$ and assume it is a root of unity: hence, as Sebastian Schoennenbeck observed, either $\zeta\in\mathbb{Q}(\xi)=\mathbb{Q}(\mu_{2p})$ or it lies in the quadratic extension $\mathbb{Q}(\mu_{4p})$. The latter is not the case since the minimal polynomial of $\zeta$ over $\mathbb{Q}(\mu_{2p})$ would be $\lambda^2-\xi$, therefore $\zeta\in \langle\xi\rangle$, the cyclic group of order $2p$ generated by $\xi$ inside $\mathbb{Z}[\xi]^\times$.
To show that this is not the case, observe that all roots of unity in $\mathbb{Z}[\xi]$ reduce to $\pm 1$ modulo the maximal ideal $\mathfrak{p}\subseteq \mathbb{Z}[\xi]$ which lies above $p$ and contains (is in fact generated by) $\xi-1$ (see Proposition 2.8 of Washingtons's Introduction to Cyclotomic Fields). If we reduce the polynomial$\mod\mathfrak{p}$ we thus find
$$
\bar{\chi}(\lambda)=\lambda^2\pm \lambda+1
$$
and neither $1$ nor $-1$ are roots of either of these polynomials unless $p=3$ which we excluded. Hence, no roots of unity can be a root of $\chi$.
The argument above extends immediately to the case when $n=p^e$ is a prime power but does not extend to the case where $n=p^aq^b$ is composite because $\xi-1$ becomes a unit. 
A: Just noticed this question.
I think the following is an even more elementary/self-contained proof.
First, let $\eta = \xi u$ (so $u$ is a root of unity iff $\eta$ is),
and divide by $\xi^2$ to get
$$
u^2 + (\xi-1+\xi^{-1}) u + 1 = 0,
$$
whose two roots are inverses of each other, so either both of them are
roots of unity or neither is.  Now since all primitive $(2n)$-th roots of
unity are conjugate, $u$ is a root of unity for one of them iff
it is a root of unity for all of them.  But then
$$
\xi - 1 + \xi^{-1} = -(u+u^{-1}) \geq -2,
$$
which holds iff $\xi = e^{2\pi i r}$ with $|r| \leq 1/3$.
But the primitive $(2n)$-th roots of unity are precisely
$e^{2 \pi i r}$ for $r=m/2n$ with $m \in [-n,n]$ coprime to $n$.
So it comes down to checking that there is such $r > 2n/3$.
But $r$ can be as large as $n-1$ or $n-2$ according as $n$ is even or odd,
and that exceeds $2n/3$ once $n>7$, QED.
Conversely, if $n$ is one of the few small numbers such that $2n$
is not coprime with any $r \in (n/3, 2n/3)$ then all conjugates of $u$
are algebraic integers of absolute value 1, and thus roots of unity
by a theorem of Kronecker; you presumably observed this for $n=5$.
