When thinking about chain homotopies in a setting involving simplices it can be helpful to consider the product $\Delta^p\times I$ where $\Delta^p$ is a $p$-simplex and $I=[0,1]$. The formula $h\sigma=\sigma_{\sharp} b_p \ast(i_p-si_p-h\partial i_p)$ corresponds to a certain inductively defined subdivision of $\Delta^p\times I$ obtained by coning off a subdivision of $\Delta^p\times\partial I \cup \partial \Delta^p \times I$ to a point in the interior of $\Delta^p\times I$. The subdivision of $\Delta^p\times\partial I \cup \partial \Delta^p \times I$ is $\Delta^p$ itself (unsubdivided) on $\Delta^p\times \{0\}$ and the barycentric subdivision of $\Delta^p$ on $\Delta^p\times\{1\}$. These are the terms $i_p$ and $si_p$ in the formula. On $\partial \Delta^p\times I$ one uses the subdivision given by induction. This is the term $h\partial i_p$. The term $\sigma_{\sharp} b_p$ corresponds to the point in the interior of $\Delta^p\times I$ that one cones off to, with the symbol $\ast$ denoting the coning operation.

What is perhaps most puzzling is that the formula says nothing about taking the product with $I$, but this is because in reality one takes the subdivision of $\Delta^p\times I$ and projects it to $\Delta^p$ before applying the map $\sigma$, whose domain is $\Delta^p$ rather than $\Delta^p \times I$. (Incidentally, you said $\sigma$ is a singular $(p+1)$-simplex in your original post but I think you meant to say singular $p$-simplex.)

I have seen this method of subdividing $\Delta^p\times I$ in several books when they are developing homology theory, but it is more complicated than necessary. A simpler subdivision that suffices is to cone off a subdivision of $\Delta^p\times \{0\} \cup \partial \Delta^p \times I$ to the barycenter of $\Delta^p\times\{1\}$, where $\Delta^p\times \{0\}$ is unsubdivided and $\partial \Delta^p \times I$ has the subdivision given inductively. On $\Delta^p\times \{1\}$ this gives just the usual barycentric subdivision, which is also defined inductively. There is a picture of this subdivision of $\Delta^p\times I$ in the case $p=2$ on page 122 of my algebraic topology book. Perhaps other books such as the Lee book you mention don't give a picture because the picture would be more complicated for the more complicated subdivision. An advantage of the simpler subdivision is that the formula for $h\sigma$ becomes just $\sigma_{\sharp} b_p \ast(i_p-h\partial i_p)$, without the term $si_p$.

The more complicated formula is given in the classic book of Eilenberg and Steenrod (page 197) without pictures or explanation. Perhaps other books are just following suit.

When thinking about chain homotopies in a setting involving simplices it can be helpful to consider the product $\Delta^p\times I$ where $\Delta^p$ is a $p$-simplex and $I=[0,1]$. The formula $h\sigma=\sigma_{\sharp} b_p \ast(i_p-si_p-h\partial i_p)$ corresponds to a certain inductively defined subdivision of $\Delta^p\times I$ obtained by coning off a subdivision of $\Delta^p\times\partial I \cup \partial \Delta^p \times I$ to a point in the interior of $\Delta^p\times I$. The subdivision of $\Delta^p\times\partial I \cup \partial \Delta^p \times I$ is $\Delta^p$ itself (unsubdivided) on $\Delta^p\times \{0\}$ and the barycentric subdivision of $\Delta^p$ on $\Delta^p\times\{1\}$. These are the terms $i_p$ and $si_p$ in the formula. On $\partial \Delta^p\times I$ one uses the subdivision given by induction. This is the term $h\partial i_p$. The term $\sigma_{\sharp} b_p$ corresponds to the point in the interior of $\Delta^p\times I$ that one cones off to, with the symbol $\ast$ denoting the coning operation.

What is perhaps most puzzling is that the formula says nothing about taking the product with $I$, but this is because in reality one takes the subdivision of $\Delta^p\times I$ and projects it to $\Delta^p$ before applying the map $\sigma$, whose domain is $\Delta^p$ rather than $\Delta^p \times I$.

I have seen this method of subdividing $\Delta^p\times I$ in several books when they are developing homology theory, but it is more complicated than necessary. A simpler subdivision that suffices is to cone off a subdivision of $\Delta^p\times \{0\} \cup \partial \Delta^p \times I$ to the barycenter of $\Delta^p\times\{1\}$, where $\Delta^p\times \{0\}$ is unsubdivided and $\partial \Delta^p \times I$ has the subdivision given inductively. On $\Delta^p\times \{1\}$ this gives just the usual barycentric subdivision, which is also defined inductively. There is a picture of this subdivision of $\Delta^p\times I$ in the case $p=2$ on page 122 of my algebraic topology book. Perhaps other books such as the Lee book you mention don't give a picture because the picture would be more complicated for the more complicated subdivision. An advantage of the simpler subdivision is that the formula for $h\sigma$ becomes just $\sigma_{\sharp} b_p \ast(i_p-h\partial i_p)$, without the term $si_p$.

The more complicated formula is given in the classic book of Eilenberg and Steenrod (page 197) without pictures or explanation. Perhaps other books are just following suit.