Given convex polygon P, the generalized incenter ICM(P) can be computed simply as follows.
Choose any "dual triangulation" of P-- that is, n-2 side-triples of P, corresponding to a triangulation of the (structural) dual of P. Any such triangulation will do (since the answer turns out to be triangulation-independent), but we get the least-tangled-looking picture if we use the dual-triangulation corresponding to the n-2 internal nodes of the medial axis of P, as shown in the following picture. Each of the n-2 chosen side-triples has an incircle, with an incenter. ICM(P) will be some weighted average of those n-2 incenters. We need to figure out the weights.
Define P' to be the tangential polygon having the unit circle as its incircle, whose angles and side directions are the same as those of P.
Define Q to be the reciprocal polygon of P' about the unit circle; that is, Q is the cyclic polygon whose vertices are the points where the sides of P' tangentially meet the unit circle. Alternatively, we can construct Q directly from P: its vertices, on the unit circle, are the outward unit normals of the sides of P.
Triangulate Q, into n-2 triangles corresponding to the n-2 chosen side-triples of P. The areas of these n-2 triangles are the desired weights, so the answer is:
$$ \mathrm{ICM(P)} = {{\Sigma\ (\mathrm{area\ of\ triangle}_i) * \mathrm{incenter}_i} \over {\Sigma\ \mathrm{area\ of\ triangle}_i}} $$
We can get a sense of the weighting by drawing each little triangle of Q with its centroid positioned at the point in P that it's weighting; then ICM(P) is the overall area centroid of the union of the little blue triangles in this picture of P.
(Note that the precise scale of the little triangles in this picture doesn't matter; e.g. we could have drawn them all twice as big, or 1/3 as big; what matters is only their centroids and their area proportions.)
Observations:
- In the case that P is a tangential polygon (i.e. it has an incircle), all of the n-2 incenters are the same, and so the result is just the incenter of P, as required.
- The result is, magically, triangulation-independent. I haven't included a proof of this.