If $G\subset GL_n({\mathbb C})$ is a compact Lie group, and $G({\mathbb C})$ is the Zariski closure of $G$ in $GL_n$, then every continuous representation of $G$ extends uniquely to an algebraic representation of $G({\mathbb C})$. Consequently, if we take $G=PU(d)$, then $G({\mathbb C})=PGL(d)=SL(d) \quad modulo \quad centre$, and the question of minimality of irreps for $G$ becomes one for $PGL(d)$. By the Borel -Weil Theorem,  every irreducible representation of $SL(d)$ is of the form $V_{\chi}=Ind_B ^G (\chi)$ where $B$ is the Borel subgroup of upper triangular matrices, and $\chi$ is an anti-dominant character. In order that this representation descend to $PGL(d)$ it is necessary and sufficient that $\chi$ be trivial on the centre of $SL(d)$. 

By Using the Weyl dimension formula, one can then see that the smallest $V_{\chi}$  is  the adjoint representation. 


I could give more details of the proof that the adjoint rep is the right one, but the "margin  is too small".