How does $E$ closed follow from the upper semicontinuity of the spectrum?

Let $$f$$ be an analytic function for a domain $$D$$ of $$\mathbb{C}$$ into a Banach algebra $$A$$. Suppose that, for all $$\lambda \in D$$, $$\text{Sp}f(\lambda)$$ is finite or a sequence converging to $$0$$. Suppose that $$\mu \neq 0$$ and $$\mu \in \text{Sp}f(\lambda_0)$$ for some $$\lambda_0 \in D$$.

Consider the set $$E = \{ \lambda \in D: \mu \in \text{Sp}f(\lambda) \}$$.

B. Aupetit mentions in a proof he writes for Theorem 3.4.26 in his book A Primer on Spectral Theory, that this set $$E$$ is closed by the upper semicontinuity of the spectrum. I am struggling to see why this is true.

Can anyone please point me in the right direction as to how I can show that $$E$$ is closed by the upper semicontinuity of the spectrum?

Upper semicontinuity of the spectrum is the following statement: if $$U$$ is a neighbourhood of $$\text{Sp}(x)$$, then there is a neighbourhood $$V$$ of $$x$$ such that $$\text{Sp}(y) \subset U$$ for all $$y \in V$$.

Suppose $$\lambda \in D \backslash E$$. That is, $$\lambda \in D$$ but $$\mu \notin \text{Sp}(f(\lambda))$$. Thus $$U = \mathbb C \backslash \{\mu\}$$ is a neighbourhood of $$\text{Sp}(f(\lambda))$$. By upper semicontinuity of the spectrum, there is a neighbourhood $$V$$ of $$f(\lambda)$$ such that $$\mu \notin \text{Sp}(y)$$ for all $$y \in V$$. Since $$f$$ is continuous, $$f^{-1}(V)$$ is a neighbourhood of $$\lambda$$, and this is disjoint from $$E$$. Thus $$D \backslash E$$ is open.