Many thanks to **Denis** for pointing out my erroneous initial "proof". This time around the proof is correct, and directly proves the assertion in line 3 of the OP, i.e., $\chi(e)\det(A)\le d_\chi(A)$ (I will write $d_\chi(I)$ instead of $\chi(e)$ for uniformity). 

The explicit notation is cumbersome, so I am just writing a proof sketch. 

 1. First, recall that $d_\chi(A)=z^T(\otimes^n A)z$ for a suitable vector $z$
 2. Next, use Cauchy-Schwarz to obtain $$|z^T(\otimes^n (X^TY))z|^2 = |z^T(\otimes^n X^T)(\otimes^n Y)z|^2\le z^T(\otimes^n X^TX)z \cdot z^T(\otimes^n Y^TY)z$$
 3. Now write $A=C^TC$ for some upper triangular matrix $C$ (since $A$ is PSD we can do this). Then, put $X=C$ and $Y=I$ above, to obtain
 4. $|z^T(\otimes^n C)z|^2 = |z^T(\otimes^n I)z|^2|\det C|^2 \le |z^T(\otimes^n C^TC)z|\cdot |z^T(\otimes^n I)z|$, where we used the upper triangular nature of $C$ for the first step. In other words, we have shown that
 5. $d_\chi(I)^2 \det(A) \le d_\chi(A)d(I)$, since $|\det C|^2=\det(C^TC)=\det(A)$.