Here are some partial answers to your question.
Let $A = A_1 \times \dots \times A_n \subseteq F^n$ be a finite grid.
Alon and Furedi proved that you need at least $\sum (\# A - 1)$ hyperplanes to cover all but one points of $A$.
If your $k$ is small enough (less than $\min \# A_i$), then you can use this bound to show that to cover all but $k$ points you need at least $\sum (\# A_i - 1) - k + 1$ hyperplanes, since by adding $k - 1$ hyperplanes you'll be covering all but one.
This seems to be a tight bound for small enough $k$.
Now let $B = B_1 \times \dots \times B_n$ be a sub-grid of $A$, i.e., $B_i \subseteq A_i$ for all $i$. S. Ball and O. Serra have proved a theorem that they called ``Punctured Combinatorial Nullstellensatz'' which can be used here to show the following:
The minimum number of hyperplanes you need to cover all points of $A$ except some point of $B$ is at least $\sum (\# A_i - \# B_i)$.
This can also be proved directly by induction on the degree of polynomial associated with the hyperplane cover, or on $\sum (\# A_i - \# B_i)$. And in fact, easier proofs of the punctured combinatorial nullstellensatz can be given.
This might give you better bounds for certain values of $k$. Especially when the $k$ points are arranged nicely.
Some other results in this general direction that can be useful are, Covering all points except one, How many $s$-subspaces must miss a point set in $PG(d , q)$.
The latter could probably give you the best possible results for the general case of covering all points but some by affine subspaces.