$\newcommand{\al}{\alpha}
\newcommand{\de}{\delta}
\newcommand{\De}{\Delta}
\newcommand{\ep}{\varepsilon}
\newcommand{\ga}{\gamma}
\newcommand{\Ga}{\Gamma}
\newcommand{\la}{\lambda}
\newcommand{\Si}{\Sigma}
\newcommand{\thh}{\theta}
\newcommand{\R}{\mathbb{R}}
\newcommand{\F}{\mathcal{F}}
\newcommand{\E}{\operatorname{\mathsf E}}
\newcommand{\PP}{\operatorname{\mathsf P}}
\newcommand{\ii}[1]{\operatorname{\mathsf I}\{#1\}}
\renewcommand{\bar}{\overline}$
The answer is yes, at least for $p\ge1$. This follows because compactly supported continuous functions are dense in $L^p$ and such functions are uniformly continuous.
Here are details. For any $f\in L^p(\R^n)$, let $\bar f$ be defined as your function $g$:
\begin{equation*}
\bar f:=\sum_{ij} \al_{f;ij}\chi_{I_{ij}},
\end{equation*}
where
\begin{equation*}
\al_{f;ij}:=\frac1{|I_{ij}|} \int_{I_{ij}} f(s)\,ds.
\end{equation*}
Take any real $\ep\in(0,1)$ and take any continuous function $f_\ep$ with a support $K_\ep\subseteq[-M_\ep,M_\ep]^n$ for some real $M_\ep>0$ such that
\begin{equation*}
\|f-f_\ep\|_p<\ep. \tag{1}
\end{equation*}
Let $$\ep_1:=\ep/(2M_\ep+2)^{n/p}.$$
By the uniform continuity of $f_\ep$, there is some real $\de_\ep>0$ such that
\begin{equation*}
|f_\ep-\bar{f_\ep}|=\sum_{ij} |f_\ep-\bar{f_\ep}|\,\chi_{I_{ij}}\le
\sum_{ij} \ep_1\chi_{I_{ij}}=\ep_1
\end{equation*}
as soon as
\begin{equation*}
\max_i(x_{i+1}-x_i)<\de_\ep. \tag{2}
\end{equation*}
Hence,
\begin{multline*}
\|f_\ep-\bar{f_\ep}\|_p^p
=\int_{\R^n} |f_\ep-\bar{f_\ep}|^p
=\int_{[-M_\ep-1,M_\ep+1]^n} |f_\ep-\bar{f_\ep}|^p \\
\le\ep_1^p(2M_\ep+2)^n=\ep^p. \tag{3}
\end{multline*}
Also, for $h:=f-f_\ep$, by Jensen's inequality and (1),
\begin{multline*}
\|\bar f-\bar{f_\ep}\|_p^p=\|\,\bar h\,\|_p^p=\int_{\R^n}|\,\bar h\,|^p
=\sum_{ij}\int_{I_{ij}}|\,\bar h\,|^p
=\sum_{ij}|\,\bar h\,|^p\,|I_{ij}| \\
\le\sum_{ij}\bar{|h|^p}\,|I_{ij}|
=\sum_{ij}\int_{I_{ij}}|h|^p
=\|h\|_p^p=\|f-f_\ep\|_p^p<\ep^p. \tag{4}
\end{multline*}
Thus, by (1), (3), (4), and Minkowski's inequality,
\begin{equation*}
\|f-\bar f\|_p\le3\ep
\end{equation*}
as soon as (2) holds.