What a fantastic question, and thanks to Asaf and Mirco for the great discussion in comments!  I love the idea of “removing” a given set from a model of ZFC, to obtain a smaller model of ZFC - some kind of inner model method analogous to the outer model method of forcing.  This may not be a complete answer, but I think that geology does offer a useful framework for attacking this question, at least when the “erosion” is strictly due to forcing (the more general question, when is a set removable *at all* while leaving behind a model of the same height, can be answered I think by looking to see whether the set in question is in $L$).

(Recall from set-theoretic geology: an inner model $W$ is a ground of our universe V if it is a transitive proper class satisfying ZFC, such there exists $G \in V$ which is generic over $W$ and $W[G]=V$. The foundational theorem of geology says the grounds of $V$ form a uniformly first-order definable collection of inner models in $V$).

Given a candidate set $A\in V$, we can ask whether $A$ is forcing-erodable by asking “Is there a ground $W$ that omits $A$”?  Any such ground $W$ is a candidate for the model obtained by removing $A$ from $V$.

How do we identify a single, canonical inner model by removing $A$?  In contrast to forcing, in which we want add as little as possible to $V$ in order to obtain $V[G]$, here we are doing the inverse - I argue that we want to remove the absolute maximum possible from $V$, while *still retaining the property that everything we remove can be added back by adding $A$ itself*.

For example, given a Cohen extension  $V[c]$, we can eliminate $c$ by going to an inner model $V[c^\prime]$ that contains only the real $c^\prime$ that lies on the even digits of $c$... but this is unsatisfying, because although we removed $c$ it feels as though we only removed half of the information contained in $c$.  To “erode $c^\prime$”, we want to go all the way down to the inner model $V$.

Geology gives us an approach.  For a set $A \in V$, call a ground $W$ of $V$ an $A$-ground if:
1. $A\notin W$ (we are eroding $A$)
2. $W[A] =V$ (we are not going ‘too far’ - everything we remove can be added back by adding $A$)

Is there a minimal such $A$-ground?  I am not certain of the answer, but the natural candidate is the intersection of all $A$-grounds (let’s call this the $A$-mantle). 

Questions:  If $M_A$ is the $A$-mantle, then 
1. is $M_A$ an $A$-ground ?  If so, this is the right candidate for “eroding $A$ from $V$”.
2. If $M_A$ is not an $A$-ground, then is $M_A$ a model of ZFC?  If that is the case, then does $M_A[A]=V$? 

These are analogous to the questions in geology “Is the Mantle a model of ZFC” and “Is the mantle necessarily a ground”.