I would suggest that you work with the Kummer surface $K$ of $J$
instead of using Mumford coordinates. The advantage is that $K$ is
a quartic surface in $\mathbb P^3$; in the case you are
considering when the curve has a unique point at infinity,
the vanishing of the first coordinate means that the point is
in the theta divisor, whereas it is the origin when the first
three coordinates vanish (using the standard Kummer coordinates
as in the book by Cassels and Flynn). Since your endomorphism
commutes with multiplication by $-1$, it induces an endomorphism
of $K$. This will be
given by a quadruple of homogeneous polynomials of some degree $d$
in the four coordinates; it should not be too hard to figure out
what they are from the generic representation in terms of the
Mumford representation. Then your problem comes down to checking
whether the first of these polynomials vanishes, and if so,
whether the next two also vanish. (This assumes that all four
polynomials do not vanish simultaneously at some point on $K$.)

When $\gamma$ is multiplication by 2, for example, the polynomials
are of degree 4 and can be obtained via

    KummerSurface(J)`Delta;

in Magma.

----------

**Added later:**

For the curve $$C \colon y^2 = x^5 + 10\,,$$ one choice
of polynomials giving multiplication by $\sqrt{5}$ on the
Kummer surface is
$$\begin{array}{r@{\,}c{\,}l}
    P_1(x_1,x_2,x_3,x_4) &=&
    8000 x_1^3 x_2^2 + 400 x_1^2 x_2 x_4^2
      + 200 x_1^2 x_3^2 x_4 + 400 x_1 x_2^2 x_3 x_4
      - 600 x_1 x_2 x_3^3 + 5 x_1 x_4^4 + 200 x_2^3 x_3^2
      + 10 x_2 x_3 x_4^3 + 10 x_3^3 x_4^2 \\
    P_2(x_1,x_2,x_3,x_4) &=&
    8000 x_1^4 x_4 + 8000 x_1^3 x_2 x_3 + 400 x_1 x_2^2 x_4^2
      + 200 x_1 x_2 x_3^2 x_4 + 400 x_1 x_3^4
      - 400 x_2^3 x_3 x_4 + 200 x_2^2 x_3^3
      - 5 x_2 x_4^4 + 10 x_3^2 x_4^3 \\
    P_3(x_1,x_2,x_3,x_4) &=&
    40 x_1^3 x_2 x_4 + 8000 x_1^3 x_3^2
      - 8040 x_1^2 x_2^2 x_3 - 200 x_1^2 x_4^3
      - 7960 x_1 x_2^4 - 996 x_1 x_2 x_3 x_4^2
      + 200 x_1 x_3^3 x_4 + 399 x_2^3 x_4^2
      - 598 x_2^2 x_3^2 x_4 - x_2 x_3^4 + 5 x_3 x_4^4 \\
    P_4(x_1,x_2,x_3,x_4) &=&
    64000 x_1^5 + 2720 x_1^3 x_3 x_4 + 8000 x_1^2 x_2^2 x_4
      + 5280 x_1^2 x_2 x_3^2 + 2720 x_1 x_2^3 x_3
      - 200 x_1 x_2 x_4^3 - 1528 x_1 x_3^2 x_4^2
      + 1600 x_2^5 - 68 x_2^2 x_3 x_4^2 - 64 x_2 x_3^3 x_4
      + 52 x_3^5 + x_4^5
  \end{array}
$$
(they are not unique, since we can add multiples of the
defining equation of the Kummer surface).

They were obtained by interpolating data from a number
of points. (Magma) Code is available from me on request.