Here is a proof for the case of a convex polytope.

If $P\subset \Bbb{R}^d$ is a compact convex polytope containing the origin $O$ in its interior then it can be broken up into cones over its facets and it suffices to verify the inequality for each of these (polyhedral) cones $C$ individually. Let $F$ be a facet of $C$ with area $A$. The hyperplane supporting $F$ separates $\Bbb{R}^d$ into two half-spaces, the inner one containing $O$ and the outer one. Then $B_{\epsilon}(F)\cap C$, the "inner" $\epsilon$-neigborhood of $F$, is contained in the inner cylinder with height $\epsilon$ based on $F$ and $B_{\epsilon}(F)\cap \overline{C}$, the "outer" $\epsilon$-neighborhood of $F$, contains the outer cylinder with height $\epsilon$ based on $F$. Therefore,

(the inner volume) $\leq \epsilon A\leq $ (the outer volume).

Here is a proof for the case of a convex polytope. (Edit: See Wlodek Kuperberg's answer for an elementary version of this argument stripped of all unnecessary notation and extended to the general case.)

If $P\subset \Bbb{R}^d$ is a compact convex polytope containing the origin $O$ in its interior then it can be broken up into cones over its facets and it suffices to verify the inequality for each of these (polyhedral) cones individually. 

Let $F$ be a facet with area $A$. The hyperplane $H$ supporting $F$ separates $\Bbb{R}^d$ into two half-spaces, the inner one, $H^{-}$, containing $O$, and the outer one, $H^{+}$. The convex cone $C=\Bbb{R}^{+}F$ is separated into the "inner" and "outer" parts, $C^{-}=C\cap H^{-}$ and $C^{+}=C\cap H^{+}$. Then $B_{\epsilon}(F)\cap C^{-}$, the "inner" $\epsilon$-neighborhood of $F$, is contained in the inner cylinder inward-facing rectangular prism with height $\epsilon$ based on $F$ and $B_{\epsilon}(F)\cap {C}^{+}$, the "outer" $\epsilon$-neighborhood of $F$, contains the outer cylinder outward-facing rectangular prism with height $\epsilon$ based on $F$. Therefore,

(the inner volume) $\leq \epsilon A\leq $ (the outer volume).