# set of centers of sphere inscribed in tetrahedron

Having a sphere and three diffrent point $A,B,C$ on this sphere. Find set of all centers of spheres inscribed in a tetrahedron $ABCD$, where $D$ is some point on the given sphere. The problem reduced to 2-dimensions is trivial it's just sum of two arcs of some circle, but in 3-dimensions the set is not so simple. Checking in geogebra it's not sum of some parts of sphere. I don't know what this set looks and how it's described.

• Why restrict $A$, $B$, and $C$ to be on the sphere? – Douglas Zare Aug 15 '15 at 16:08
• The boundary of the set is formed by incenters of tetrahedra with $D$ at infinity. Do not expect it to be particular nice surface. – Anton Petrunin Aug 15 '15 at 16:33
• $D$ is on the same sphere as $A,B,C$. It makes that the set is inside that sphere – M.Martin Aug 15 '15 at 19:29
• @AntonPetrunin: "with $D$ at infinity": Could you expand on your comment? – Joseph O'Rourke Aug 15 '15 at 21:39
• @JosephO'Rourke, ignore it, it was a comment to the old version of the question where $D$ is arbitrary. – Anton Petrunin Aug 16 '15 at 14:15

A little algebra shows that, for $A=(1,0,0)$, $B=(0,1,0)$, and $C=(0,0,1)$ on the sphere, this surface is an irreducible algebraic surface of degree 5 that is singular at $A$, $B$, and $C$ but is otherwise smooth (even on the plane at infinity). In fact, the hypersurface is defined by \begin{aligned} 0 &= 2s_1^5-9s_1^3s_2+9s_1^2s_3+9s_1s_2^2-27s_2s_3\\ &\quad -7s_1^4+18s_1^2s_2+18s_1s_3-9s_2^2\\ &\quad\quad +8s_1^3-9s_1s_2-27s_3 -2s_1^2 -2s_1 +1 \end{aligned} where $s_1=x+y+z$, $s_2=xy+yz+zx$, and $s_3=xyz$. (Written out in terms of $x$, $y$, and $z$, the polynomial on the right hand side has 56 terms; as Anton suspected, it does not appear to be very nice.)

The singularities at $A$, $B$, and $C$ are cubic (i.e., the polynomial, when expanded about each of these points, has lowest nonvanishing terms of order $3$), so one suspects that this surface, once the singularities have been resolved, might be a known algebraic surface. In fact, the projectivization of the tangent cone at each of these three points is a nonsingular cubic curve. (I have not computed the j-invariant, though.)

For $A,B,C=$ $(1,0,0)$, $(0,1,0)$, $(0,0,1)$ (blue below) on a unit sphere $S$, the surface is a sort of triangular tea bag with corners at $A,B,C$. Below are two views of $100$ random tetrahedron incenters (red) on the surface, corresponding to random points $D$ (not shown) uniformly distributed on $S$.

A typical $D$ (green), the determined tetrahedron, the inscribed sphere and its (red) center, are shown below.

Here is a depiction of Robert Bryant's surface defined by the $56$-term polynomial he details:

Note one component is the "triangular tea bag" discernable in my empirical investigation. Here is the polynomial: $$2 x^5+x^4 y+x^4 z-7 x^4+2 x^3 y^2+4 x^3 y z-10 x^3 y+2 x^3 z^2-10 x^3 z+8 x^3+2 x^2 y^3-12 x^2 y^2 z-15 x^2 y^2-12 x^2 y z^2+6 x^2 y z+15 x^2 y+2 x^2 z^3-15 x^2 z^2+15 x^2 z-2 x^2+x y^4+4 x y^3 z-10 x y^3-12 x y^2 z^2+6 x y^2 z+15 x y^2+4 x y z^3+6 x y z^2-6 x y z-4 x y+x z^4-10 x z^3+15 x z^2-4 x z-2 x+2 y^5+y^4 z-7 y^4+2 y^3 z^2-10 y^3 z+8 y^3+2 y^2 z^3-15 y^2 z^2+15 y^2 z-2 y^2+y z^4-10 y z^3+15 y z^2-4 y z-2 y+2 z^5-7 z^4+8 z^3-2 z^2-2 z+1$$ The plot above restricts $(x,y,z)$ to lie on or in the sphere.

• Thanks for making the drawing. In case others might wonder what the other 'components' are, here is a comment: What the algebraic surface describes is the locus of centers of spheres that are tangent to all four planes that make up the sides of a tetrahedron ABCD where D is an arbitrary point on the 2-sphere. Of course, one such sphere will be the sphere that lies in the convex hull of the 4 points, but, of course, there are four others. The polynomial equation can't distinguish these. As your picture shows, even requiring that the center be interior to the original sphere is not enough. – Robert Bryant Aug 17 '15 at 3:31