We do not need the pentagon to be cyclic. But we do have a requirement for the angles.
Begin with two cases where a cyclic path can be defined. In the first case it does not exactly meet the definition of a billiard path because the sides are not hit consecutively. In the second, however, the full definition of a billiard path is satisfied without using a cyclic pentagon.
Case 1
The mirror-symmetric pentagon below is clearly not cyclic (the top vertex lies outside the circle that passes through the other four) but has the self-crossing cyclic path indicated in blue. The angles of the pentagon measure $45°,157.5°,90°,90°,157.5°$ in clockwise order from the top, and the bottom side is congruent to its neighbors.
Case 2
To get the reflections in proper order from the above case, we rotate the top two sides $45°$ and shorten the right and left vertical sides to prevent the ball from reaching the crossing point seen in Case 1. Using a 3:5 ratio for the vertical to the horizontal sides and vertex angles now measuring $135°,112.5°,90°,90°,112.5°$ we get the figure below.
The top vertex falls inside the circle passing through the other four (putting all five vertices on the same circle would have required keeping the vertical sides as long as the horizontal one), and so we now have a full-fledged billiard path within a noncyclic pentagon.
Both cases above are among the simplest representations of a cyclic path where the bounding polygon is neither inscribed in nor circumscribed around a circle.
What's your angle?
Why does a triangle have to be acute to admit a billiard circuit? Imagine that you have a triangle $ABC$ with angles $\theta_A,\theta_B,\theta_C$ at these respective vertices. Define $\psi_A$ as the angle in the billiard circuit when it hits the side opposite $A$, and similarly for $\psi_B$ and $\psi_C$. The sides of the billiard circuit and those of the bounding triangle form smaller triangles whose angles sum to $180°$. That plus the equal-angle law for reflection leads to the following system of equations:
$\psi_A+\psi_B=2\theta_C$
$\psi_B+\psi_C=2\theta_A$
$\psi_C+\psi_A=2\theta_B$
$\theta_A+\theta_B+\theta_C=180°$
from which we can readily solve for the angles of the billiard circuit. Thus
$\psi_A=180°-2\theta_A$
$\psi_B=180°-2\theta_B$
$\psi_C=180°-2\theta_C$
To make a proper polygon for the billiard circuit in a bounding triangle, the $\psi$ angles must be positive, forcing the angles of the bounding triangle to indeed be all less than $90°$.
We can do the same thing with a pentagon. Let $\theta_A$ through $\theta_E$ be the angles in the bounding pentagon in rotational order, and $\psi_A$ through $\psi_E$ be the corresponding opposite angles on the billiard circuit (e.g. $\psi_A$ is on side $CD$ opposite vertex $A$). We then have
$\psi_A+\psi_B=2\theta_D$
and cyclic permutations. Also the bounding angles $\theta_A$ through $\theta_E$ sum to $540°$. The solution by standard methods for systems of linear equations is given by
$\psi_A=540°-2(\theta_B+\theta_E)$
and, again, cyclic permutations. Since $\psi_A$ has to lie strictly between $0°$ and $180°$ thus means the sum of the two nonadjacent vertex angles of the bonding pentagon, $\theta_B+\theta_E$, must lie strictly between $180°$ and $270°$. To get the rest of the billiard circuit angles in range the other four pairs of nonadjacent angles must mert a similar requirement.
We can see this constraint at work in the two pentagonal cases described above. In the first one we had to make the circuit cross itself and hit the sides in nonrotational order. Now we see that two of the nonadjacent angles, each measuring $157.5°$, add up to $315°$ — no good. Other pairs of nonadjacent angles in the Case 1 pentagon add up to only $135°$. Case 2 is different. The reader can verify that with the angles given for that pentagon ($135°,112.5°,90°,90°,112.5°$ in rotational order), all pairs of adjacent angles add up to either $202.5°$ or $225°$, thus in range; and we see the existence of a proper billiard circuit.
Not all pentagons meeting this angular constraint will admit such a circuit; there must be an additional requirement. It is conjectured here, based on the side-length constraints required for the Case 2 pentagon to have its billiard circuit, that (perhaps among other things) the circumcenter of every triangle formed by three adjacent vertices lies inside the pentagon.
