Yes, this is always true if $\lambda \not= 0$. The subsequent theorem shows a more general result. To formulate it, we need the following terminology:

- For an eigenvector $\lambda$ of a bounded linear operator $T: X \to X$ on a complex Banach space $X$, the vector subspace $\bigcup_{n \ge 1} \ker\big((\lambda-T)^n\big)$ of $X$ is called the *generalized eigenspace* of $T$ for the eigenvalue $\lambda$. 
  
  The eigenvalue $\lambda$ is called *semi-simple* if its generalized eigenspace is equal to the eigenspace $\ker(\lambda - T)$. It is not difficult to show by induction that $\lambda$ is semi-simple if and only if $\ker((\lambda-T)^2) = \ker(\lambda-T)$.

- Let $X$ be a Banach space. Two vector subspaces $V \subseteq X$ and $V' \subseteq X'$ (where $X'$ denotes the norm dual space of $X$) are said to *separate each other* if for every non-zero $v \in V$ there exists $v' \in V'$ such that $\langle v', v \rangle \not= 0$, and vice versa. 
  
  If $P: X \to X$ is a bounded linear projection, then it is easy to see that the ranges of $P$ and the dual operator $P'$ (which is also a projection) separate each other. Indeed, for every non-zero $x \in PX$ we can find a functional $x' \in X'$ such that 
  $$
    0 \not= \langle x',x \rangle = \langle x', Px \rangle = \langle P'x', x \rangle,
  $$ 
  so $P'x'$ is an element of the range of $P'$ which does not vanish on $x$. A similar argument shows the other direction.
  
  Of course, two non-zero vectors $x \in X$ and $x' \in X'$ satisfy $\langle x', x \rangle \not= 0$ if and only if their spans seperate each other.

Here is a general result about separation of eigenspaces and dual eigenspaces. I formulate it in the Banach space setting, with duals of operators rather than adjoints. But it can be readily transferred to the Hilbert space setting by using essentially the same proof and replacing the dual eigenvalue $\lambda$ with $\overline{\lambda}$.

**Theorem.** Let $\lambda \not= 0$ be a semi-simple eigenvalue of compact operator $T: X \to X$ on a complex Banach space $X$. Then the eigenspaces $\ker(\lambda - T)$ and $\ker(\lambda - T')$ separate each other. 

*Proof.* As $T$ is compact and $\lambda$ is non-zero, it follows that $\lambda$ is isolated in the set $\sigma(T) = \sigma(T')$. The resolvent $R(\cdot,T)$ which is defined as 
$$
  R(\mu,T) := (\mu \operatorname{\operatorname{id}} - T)^{-1}
$$
for every $\mu$ in the resolvent set $\rho(T)$ of $T$, is a holomorphic mapping from the open set $\rho(T)$ into the space $\mathcal{L}(X)$ of bounded linear operator on $X$ (endowed with the operator norm). 
The number $\lambda \not= 0$ is an isolated singularity of the resolvent, and as $T$ as compact, $\lambda$ is even a pole of the resolvent, say of order $p \ge 1$ (the order cannot be $0$ as we assumed $\lambda$ to be an eigenvalue and thus a spectral value of $T$).
Hence, $R(\cdot,T)$ has a Laurent series expansion about $\lambda$ with finite principal part, i.e.,
$$
  \tag{$*$}
  R(\mu,T) = \sum_{k=-p}^\infty (\mu - \lambda)^k Q_k
$$
(with operators $Q_k \in \mathcal{L}(X)$) for all a $\mu$ in a small pointed neighbourhood of $\lambda$ (where the series converges absolutely). 
The Laurent coefficient $P := Q_{-1}$ is a projection - the so-called *spectral projection* of $T$ for the spectral value $\lambda$ (it can also be obtained by means of a contour integral around $\lambda$). 

Now we need the following general fact about poles of the resolvent of linear operators: the range of the spectral projection is always the generalized eigenspace. In our case, we assumed the eigenvalue to be semi-simple; hence, the range of the projection $P$ is equal to the eigenspace $\ker(\lambda-T)$.

Finally, we will show that the dual eigenspace is the range of the dual operator $P'$, which implies (as observed before the theorem) that $\ker(\lambda-T) = PX$ and $\ker(\lambda-T') = \ker P'X'$ separate each other. This can be done as follows:

One can check that a pole of the resolvent (which is, by the way always, an eigenvalue, no matter whether $T$ is compact or not - since it can be shown that the range of the leading Laurent coefficient is a subspace of the eigenspace) is a semi-simple eigenvalue if and only if the pole order is $1$. 
Hence, in our case we have $p=1$.

By dualizing the Laurent expansion $(*)$ we see that the order of $\lambda$ as a pole of $R(\cdot,T)$ coincides with its order as a pole of $R(\cdot,T') = R(\cdot,T)'$. 
Thus, the dual resolvent also has a pole of order $1$ at $\lambda$, and therefore $\lambda$ is also a semi-simple eigenvalue of the dual operator $T'$. 
Moreover, it also follows from dualizing the Laurent expansion $(*)$ that $P'$ is the spectral projection of the dual operator $T'$ for the eigenvalue $\lambda$. Hence, the dual eigenspace $\ker(\lambda-T')$ indeed coincides with the range $P'X'$. $\square$

**Remark.** The proof of the theorem shows that we do not need $T$ to be compact; it suffices if $\lambda$ is a pole of the resolvent. Moreover, one does not need $T$ to be bounded, either: All arguments remain the same if $T$ is a closed linear operator with non-empty resolvent set, as long as $\lambda$ is a pole of the resolvent (which is, for instance, always true if $T$ has compact resolvent).

**Literature.** While all the arguments used above are essentially classical results in spectral theory, it is surprisingly hard to find references where all the results needed for such arguments are stated explicitly. Most of it is present somewhere in Yosida's [Functional Analysis](https://zbmath.org/?q=an%3A0435.46002) or Kato's [Perturbation theory for linear operators](https://zbmath.org/?q=an%3A0342.47009), but in a very implicit form - i.e., the results that one wants to use are consequenes of theorems in those books, rather than stated there explicitly. In my experience, it's quite hard in these books to find the results that one wants to use, unless one already knows what precisely is true and where precisely to look for it.

As I found this quite annoying, I wrote a very concise summary of these types of spectral theory results in Appendix A of my [PhD thesis](http://dx.doi.org/10.18725/OPARU-4238). The appendix contains only very few detailed proofs, but it gives precise references to show how the various results can be derived from theorems in classical monographs such as Kato's or Yosida's. 

I also find the introduction to spectral theory in Chapter IV of Engel and Nagel's [One-parameter semigroups for linear evolution equations](https://zbmath.org/?q=an%3A0952.47036) very useful, as it also collects and consolidates various useful spectral theory results from the literature.