This is maybe more addressed at the comments and the discussion with Timothy Chow, but I just wanted to point out that, at least in a certain context, there is a very very "concrete" description of the $h$-vector. Say $\mathcal{P}$ is a simple (convex, full-dimensional) polytope in $\mathbb{R}^n$. Then let $\phi$ be a generic enough linear functional on $\mathbb{R}^n$. Use $\phi$ to orient the $1$-skeleton of $\mathcal{P}$: orient an edge $uv$ from $u$ to $v$ if $\phi(u) < \phi(v)$ (since $\phi$ is generic there will not be ties). Then if $h=(h_0,h_1,\ldots,h_n)$ is the $h$-vector of $\mathcal{P}$ (defined in the usual way as a transform of the $f$-vector), we have that $$ h_i = \# (\textrm{vertices $v$ with indegree $=i$})$$ according to our orientation of the $1$-skeleton. So for instance this explains that the $h_i$ are positive, that $h_0+h_1+\cdots+h_n$ is the number of vertices; also we will have a $h_i=h_{n-i}$ symmetry which swaps indegree according to $\phi$ for outdegree according to $-\phi$, etc.
Incidentally, I don't know who to attribute this simple but nice perspective on the $h$-vector to; to me it is folklore.
EDIT: As Richard notes in the comments this perspective is the same as the idea of a line shelling for a simplicial polytope, which I guess was assumed by Schläfli in his proof of the Euler-Poincaré formula and formally established by Bruggesser and Mani.