So, the latest formulation is much better, but there are still some problems with this setup. You can push-forward cycles via morphisms, but rational maps are trickier. 

First of all if the inverse of $f$ contracts a divisor, then $f_*$ does not (necessarily) respect linear equivalence. Here is an example. Let $X$ be your favorite smooth projective variety and $\pi:Y\to X$ the blow-up of a (closed) point $x\in X$. Let $f:X\dashrightarrow Y$ be the inverse of $\pi$ (as a rational map). Now let $\mathfrak d$ be a very ample linear system on $X$. If $D\in\mathfrak d$ is such that $x\not\in X$, then $f_*D\sim\pi^*D$ and if  $D'\in\mathfrak d$ is such that $x\in D'$ and, for simplicity, the multiplicity of $D'$ at $x$ is $1$ (this happens for instance if $X=\mathbb P^n$ and $\mathfrak d$ is the hyperplane class), then $f_*D'=\pi^*D'-E \sim \pi^*D-E\sim f_*D-E$. 

Therefore there is a short exact sequence of sheaves on $Y$:
$$
0\to \mathscr O_Y(f_*D')\to \mathscr O_Y(f_*D) \to \mathscr O_E \to 0.
$$
Taking global sections one obtains another short exact sequence:
$$
0\to H^0(Y,\mathscr O_Y(f_*D'))\to H^0(Y,\mathscr O_Y(f_*D)) \to H^0(E,\mathscr O_E) \to 0.
$$
Here the non-obvious exactness follows from the fact that $f_*D\cap E=\emptyset$ and hence the restriction of the corresponding section to $E$ is a non-zero global section of $\mathscr O_E$.  

This shows that (using the OP's notation): 
$$
h^0(f_*D')= h^0(f_*D)-1.
$$
On the other hand, from the projection formula it follows that
$$
h^0(D)=h^0(f_*D)
$$
and hence
$$
h^0(D')=h^0(f_*D')+1
$$

----

OK, so we see that we better assume that the inverse of $f$ does not contract divisors. I assume you meant this to be part of the assumption that $f$ is a *small modification*, but you also seemed to be asking without that. Also, the above should warn you that using notation like $h^0(f_*D)$ is dangerous. The usage of $h^0$ suggests using the linear equivalence *class* of $D$, but $f_*D$ is not well-defined for that. Of course, this can still be done right, you just have to make sure to emphasize that $D$ is an explicit divisor, not a divisor class. (So for instance this is another reason why your original $D\in\mathrm{Pic}\\, X$ was bad).


Assuming that $f^{-1}$ does not contract divisors, there is of course a problem if $f$ contracts a divisor. Just take $D''=f_*D'$ from the above example. Clearly, $\pi_*D''=D'$ and we already saw that 
$$
h^0(D'')=h^0(\pi_*D'')-1.
$$

----

So, we're left with the case when neither $f$ nor $f^{-1}$ contract any divisors. In this case
it is indeed true what you want. Here is why:

The assumption means that in this case we have open sets $U\subseteq X$ and $V\subseteq Y$ such that  $\mathrm{codim}_X(X\setminus U)\geq 2$, $\mathrm{codim}_Y(Y\setminus V)\geq 2$, and $f:U\to V$ is an isomorphism. In this case clearly we have that 
$$
H^0(X,\mathscr O_X(D)) = H^0(U,\mathscr O_U(D|_U)) = H^0(V,\mathscr O_V(f(D|_U))) = H^0(Y,\mathscr O_Y(f_*D)).
$$
The middle equality is obvious the other two follows from the fact that $X$ and $Y$ are $S_2$. This is sometimes called the Hartog property. See [this MO answer][1] for more.

--- 

Note that this does not need $X$ and $Y$ to be smooth, only $S_2$. In order to deal with divisors you're probably better off assuming that they are normal. For some musings about that see [this MO answer][2].

Also, to be fair, you asked for a relation in general, not equality so I assume you are aware of some of the above. I think that in general the relationship between $h^0(D)$ and $h^0(f_*D)$ will be very complicated and has to do with how $D$ relates to the exceptional divisor(s).


  [1]: http://mathoverflow.net/questions/45347/why-does-the-s2-property-of-a-ring-correspond-to-the-hartogs-phenomenon/45354#45354
  [2]: http://mathoverflow.net/questions/35736/the-canonical-line-bundle-of-a-normal-variety/46663#46663