Let $S_n$ be the set of all permutations of $X$ \(i.e. $S_n = \{ f$ $|$ $f : X \rightarrow X\}$). Now consider the set of permutations, $A$, that have exactly k fixed points. More formally, $A$ is the union of all sets
$$A_i = \{f \in S_n : N' \in {\{1,...,n\} \choose k}, f(N') = N', \text{ and } \forall j \in \{1,...,n\} \setminus N', f(x_j) \neq x_j\}$$ where $i \in \{1,...,{n \choose k}\}$.
(Note that each $A_i$ must uniquely correspond to some $N' \in {\{1,...,n\} \choose k}$, which is why $i \in \{1,...,{n \choose k}\}$. In other words, the definition of $A_i$ implicitly defines a bijection between all $A_i$ and ${\{1,..., n\} \choose k}$).
Since there are $k$ fixed elements for all $f \in A$, we consider how to permute the remaining $n - k$ elements. These elements cannot be fixed. Thus, by letting arbitrary $N' \in {\{1,...,n\} \choose k}$, the number of permutations of $X$ where the $k$ points in $N'$ are fixed and the rest are not is equivalent to the number of derangements (see side note for further explanation) of the set $X \setminus N'$. Additionally, because there are ${n \choose k}$ ways to fix $k$ elements, we have
$${n \choose k} \cdot D(|X \setminus N'|) = {n \choose k} \cdot D(n - k) = {n \choose k} \cdot ((n - k)! - \displaystyle\sum_{i = 1}^{n - k} (-1)^{i - 1} {(n - k) \choose i} (n - k - i)!)$$
as the total number of permutations of $X$ with exactly $k$ fixed
points. More concisely, we have $${n \choose k} \cdot D(|X \setminus N'|) = {n \choose k} ((n - k)! - \displaystyle\sum_{i = 1}^{n - k} (-1)^{i - 1} \dfrac{(n - k)!}{i!})$$
Side note: A derangement of a set is a permutation of the set such that no element is mapped to itself. The total number of derangements of an n-element set is $$D(n) = n! - \displaystyle\sum_{i = 1}^n (-1)^{i - 1} {n \choose i}(n - i)! = n! - \displaystyle\sum_{i = 1}^n (-1)^{i - 1} \dfrac{n!}{i!}$$
The proof for the total number of derangements of a set utilizes the principle of inclusion-exclusion, but I will not include it here as it does not directly answer your question.