The reason is that there are two ways of thinking about "points".

Let $A$ be a ring. Then, define:

1. A scheme-theoretic/topological point of Spec $A$ is a prime ideal of $A$.
2. A geometric/functorial point of Spec $A$ is an equivalence class of morphisms Spec $K\rightarrow$ Spec $A$, where $K$ is a field, and two morphisms:
$$p_1 : \text{Spec }K_1\rightarrow \text{Spec }A,\qquad p_2 : \text{Spec }K_2\rightarrow\text{Spec }A$$
are equivalent if there exists a field $K$ containing both $K_1,K_2$ as subfields, and a morphism Spec $K\rightarrow$ Spec $A$ making the obvious diagrams commute.

Note that if $A$ is a $k$-algebra, then geometric points of Spec $A$ are just morphisms Spec $\overline{k}\rightarrow$ Spec $A$.

Here's the example that will explain everything:

Let $k = \mathbb{Q}$, and let $A = \mathbb{Q}[x]$, $B = \mathbb{Q}[y]$, and suppose $X = \text{Spec }A$ and $Y = \text{Spec }B$. Consider the map $f : Y\rightarrow X$ given by $x\mapsto y^2$. This map is degree 2, and sends $y = a$ in $Y$ to $x = a^2$ in $X$.

First, lets consider the scheme-theoretic/topological point $q\in X$ corresponding to $x = -1$. Thus, $q$ is the prime ideal $(x+1)\subset\mathbb{Q}[x]$. It's not hard to check that the only prime of $B$ lying over $q$ is $p := (x^2+1)$, whose residue field is $\mathbb{Q}(i)$, so $p$ has "inertia degree" 2 over $q$.

Now, lets consider the geometric point $Q : $Spec $\overline{\mathbb{Q}}\rightarrow$ Spec $A$ given by sending $x\mapsto -1$. It's not hard to see that sending $y\mapsto i$ and sending $y\mapsto -i$ define two distinct morphisms $P_1,P_2 : \overline{\mathbb{Q}}\rightarrow$ Spec $B$ (ie, two distinct geometric points) which lie above $Q$.

Thus, while there is only one topological point above $x = -1$, there are two geometric points above it.

Now, note that a geometric point of (for example) Spec $k[x,y]/(f)$ is nothing but a pair $(a,b)\in\overline{k}^2$ satisfying the equation $f$, which is precisely how Silverman defines a "point".

Thus, while there is only one topological point above $x = -1$, there are two geometric points, each of which makes an appearance as a summand of the equation in Prop 2.6. This makes up for the fact that the inertia degree doesn't appear. In fact, one may *define* the inertia degree as the number of geometric points lying over $x = -1$ who have the same image $P\in Y$.

Ie, in the situation of our morphism $f : Y\rightarrow X$, the two equivalent ways of writing the formula would be:

1. $deg(f) = 2\cdot e_f(p) = 2\cdot 1$ (2 is the inertia degree), or
2. $deg(f) = e_f(P_1) + e_f(P_2) = 1 + 1$