For matrices $X,Y\in [0,1]^{n\times m}$, for n > m, is there a square matrix $W\in R^{n\times n}$ so that $X^TWY$ is diagonal if and only if $Y = X$? Furthermore, $X$ and $Y$ are column normalized so that $X1_m = Y1_m = 1_m$, where $1_m$ is the m-length column vector with all entries equal to 1.
I know that if $X$ and $Y$ are binary, then $W=I$.
UPDATE Sorry, previous version was wrong. For $n=m=2$, this computation shows that there is such a matrix. FURHTER UPDATE However, for $m=n=3$, it shows there isn't.
Taking $n=m=2$, we see that $X$ and $Y$ are of the form $\begin{pmatrix} x & 1-x \\ 1-x & x \end{pmatrix}$ and $\begin{pmatrix} y & 1-y \\ 1-y & y \end{pmatrix}$. Set $W = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$ Then $X^T W Y =: \begin{pmatrix} e & f \\ g & h \end{pmatrix}$ with
$$f=c+(a-c)x+(d-c) y+(-a+b+c-d)xy$$ $$g=b+(a-b)x+(d-b) y+(-a+b+c-d)xy$$
Your desire is that we have $f(x,y) = g(x,y) =0$ iff $x=y$. Plugging in $x=y$, we deduce that we must have $b=c=a+d=0$. But then $f$ and $g$ vanish for all $x$ and $y$. And, indeed, $(a,b,c,d) = (1,0,0,-1)$ solves the problem for $m=n=2$.
Now run the same analysis with $m=n=3$, thinking about $X$ and $Y$ of the form $\begin{pmatrix} x & 1-x & 0 \\ 1-x & x & 0 \\ 0 & 0 & 1 \end{pmatrix}$. We deduce that $w_{12} = w_{21}=0$ and $w_{11}=-w_{22}$. Similarly, $w_{22} = - w_{33}$, $w_{11} = - w_{33}$ and $w_{13}=w_{31} = w_{23} = w_{32}=0$. But the only solution to these linear equations is $W=0$..
The general $n=m$ case: First take $X$ to be the identity, this gives us that $W$ is the diagonal. Then for each pair of entries on the diagonal, look at the set of matrices that are almost entirely the identity, but have a 2$\times$2 block that looks like one of David Speyer's 2$\times$2 matrices and contains those two diagonal entries. Take $X$ and $Y$ be one of those matrices, then his calculations show that these two diagonal entries are minus each other. So if $n\geq 3$ then each pair of entries sum to $0$, so all sum to $0$, but this is also impossible, a contradiction.
The general $n>m$ case. Let $X$ be an $n\times n$ matrix. Consider all matrices $X'$ formed by removing $n-m$ rows, and consider $X'^TWX'$. These must all be diagonal, so if $m\geq 2$ then $X^TWX$, since every off-diagonal entry in it is an off-diagonal entry in one of these matrices. So we reduce to the previous case.
Therefore, $n\geq 3$ and $m\geq 2$ is unsolvable. (building on David Speyer's answer). By an obvious extension of Arthur B's answer, $n>m=1$ is unsolvable. This, combined with David's observation that $n=m=2$ is solvable, answers every case.
In general no, take n = m = 1
Here is a solution: $W = X(X^TX)^{-1}D(X^TX)^{-1}X^T$ for a diagonal matrix $D$. You may verify that $X^TWY = D$ iff $Y = X$.
