In general $\rho^\vee$ is an element of the co-weight lattice. The notation $\rho^\vee(-1)$ only makes sense if $\rho^\vee$ is in fact in the co-character lattice, i.e. the lattice $\text{Hom}(\mathbb G_m,T)$.
This holds, for example, if $G$ is adjoint.
If this condition holds, $\rho^\vee(s)$ is an element of $T$ satisyfing $\alpha(\rho^\vee(s))=s^{\langle \alpha,\rho^\vee\rangle}$ as you said (this holds for all roots). However this condition only determines an element of $T$ up to the center (the center of $G$ is precisely the kernel of all of the roots).
The precise value of $\rho^\vee(-1)$ as a matrix requires some case-by-case analysis, although for most simple groups the center is very small and it isn't hard.
For $SL(n)$ with the usual choice of positive roots
$\rho^\vee=\frac12(n-1,n-2,\dots, -n+1)$. This is in the co-character lattice if and only if $n$ is odd. In this case:
$$
\begin{aligned}
\rho^\vee(-1)&=\exp(\pi i\rho^\vee)\\
&=(i^{n-1},i^{n-3},\dots, i^{-n+1})\\
&=(-1,1,\dots,1,-1)
\end{aligned}
$$
If $n$ is even $\rho^\vee(-1)$ simply doesn't make sense.
Addendum: For a complex group you can use the exponential map, which satisfies $\gamma^\vee(e^z)=\exp(z\gamma^\vee)$.
An adjoint group is one with trivial center. Equivalently the co-weight lattice is equal to the co-character lattice.