Return to Answer

Not an answer, just two remarks.

(1) Literature. There is an old paper that finds a largest inscribed equilateral triangle, and a largest inscribed square, but these results do not straightforwardly generalize:

DePano, A., Yan Ke, and J. O’Rourke. "Finding largest inscribed equilateral triangles and squares." In Proc. 25th Allerton Conf. Commun. Control Comput, pp. 869-878. 1987.

Unfortunately, I no longer have access to my own article. :-/ My recollection is that we achieved $O(n^3)$ time for equilateral triangles and $O(n^2)$ time for squares, for a convex polygon of $n$ vertices.

---

     
^{See [Mathworld](http://mathworld.wolfram.com/EquilateralTriangle.html): In unit square, side $s=\sec(15^\circ)$ and area $A=2\sqrt{3}-3$.}

---

Another relevant paper from the same period:

Fekete, Sandor P. "Finding all anchored squares in a convex polygon in subquadratic time." (1992). Download author's PDF.

(2) Question 3: "does $A(n)$ have exactly one local maximum (which is also the global max) for any $C$?"

No, not always. Let $C$ be an equilateral triangle. Then the global max is $A(3)$, but $A(6)$ is a local max:

---

          [![Hex_in_Tri][1]][1]
          ^{$A(3) > A(4) < A(6) > A(12)$.}

---

So perhaps you should exclude $C$ being a regular polygon itself.

Not a full answer, just two remarks.

(1) Literature. There is an old paper that finds a largest inscribed equilateral triangle, and a largest inscribed square, but these results do not straightforwardly generalize:

DePano, A., Yan Ke, and J. O’Rourke. "Finding largest inscribed equilateral triangles and squares." In Proc. 25th Allerton Conf. Commun. Control Comput, pp. 869-878. 1987.

Unfortunately, I no longer have access to my own article. :-/ My recollection is that we achieved $O(n^3)$ time for equilateral triangles and $O(n^2)$ time for squares, for a convex polygon of $n$ vertices.

---

     
^{See [Mathworld](http://mathworld.wolfram.com/EquilateralTriangle.html): In unit square, side $s=\sec(15^\circ)$ and area $A=2\sqrt{3}-3$.}

---

Another relevant paper from the same period:

Fekete, Sandor P. "Finding all anchored squares in a convex polygon in subquadratic time." (1992). Download author's PDF.

(2) Question 3: "does $A(n)$ have exactly one local maximum (which is also the global max) for any $C$?"

No, not always. Let $C$ be an equilateral triangle. Then the global max is $A(3)$, but $A(6)$ is a local max:

---

          [![Hex_in_Tri][1]][1]
          ^{$A(3) > A(4) < A(6) > A(12)$.}

---

So perhaps you should exclude $C$ being a regular polygon itself.

Not an answer, just two remarks.

(1) Literature. There is an old paper that finds a largest inscribed equilateral triangle, and a largest inscribed square, but these results do not straightforwardly generalize:

DePano, A., Yan Ke, and J. O’Rourke. "Finding largest inscribed equilateral triangles and squares." In Proc. 25th Allerton Conf. Commun. Control Comput, pp. 869-878. 1987.

Unfortunately, I no longer have access to my own article. :-/ My recollection is that we achieved $O(n^3)$ time for equilateral triangles and $O(n^2)$ time for squares, for a convex polygon of $n$ vertices.

---

     
^{See [Mathworld](http://mathworld.wolfram.com/EquilateralTriangle.html): In unit square, side $s=\sec(15^\circ)$ and area $A=2\sqrt{3}-3$.}

---

Another relevant paper from the same period:

Fekete, Sandor P. "Finding all anchored squares in a convex polygon in subquadratic time." (1992). Download author's PDF.

(2) Question 3: "does $A(n)$ have exactly one local maximum (which is also the global max) for any $C$?"

Let $C$ be an equilateral triangle. Then the global max is $A(3)$, but $A(6)$ is a local max:

---

          [![Hex_in_Tri][1]][1]
          ^{$A(3) > A(4) < A(6) > A(12)$.}

---

So perhaps you should exclude $C$ being a regular polygon itself.

Not an answer, just two remarks.

(1) Literature. There is an old paper that finds a largest inscribed equilateral triangle, and a largest inscribed square, but these results do not straightforwardly generalize:

DePano, A., Yan Ke, and J. O’Rourke. "Finding largest inscribed equilateral triangles and squares." In Proc. 25th Allerton Conf. Commun. Control Comput, pp. 869-878. 1987.

Unfortunately, I no longer have access to my own article. :-/ My recollection is that we achieved $O(n^3)$ time for equilateral triangles and $O(n^2)$ time for squares, for a convex polygon of $n$ vertices.

---

     
^{See [Mathworld](http://mathworld.wolfram.com/EquilateralTriangle.html): In unit square, side $s=\sec(15^\circ)$ and area $A=2\sqrt{3}-3$.}

---

Another relevant paper from the same period:

Fekete, Sandor P. "Finding all anchored squares in a convex polygon in subquadratic time." (1992). Download author's PDF.

(2) Question 3: "does $A(n)$ have exactly one local maximum (which is also the global max) for any $C$?"

No, not always. Let $C$ be an equilateral triangle. Then the global max is $A(3)$, but $A(6)$ is a local max:

---

          [![Hex_in_Tri][1]][1]
          ^{$A(3) > A(4) < A(6) > A(12)$.}

---

So perhaps you should exclude $C$ being a regular polygon itself.

Not an answer, just a remark. There is an old paper that finds a largest inscribed equilateral triangle, and a largest inscribed square, but these results do not straightforwardly generalize:

DePano, A., Yan Ke, and J. O’Rourke. "Finding largest inscribed equilateral triangles and squares." In Proc. 25th Allerton Conf. Commun. Control Comput, pp. 869-878. 1987.

Unfortunately, I no longer have access to my own article. :-/ My recollection is that we achieved $O(n^3)$ time for equilateral triangles and $O(n^2)$ time for squares, for a convex polygon of $n$ vertices.

---

     
^{See [Mathworld](http://mathworld.wolfram.com/EquilateralTriangle.html): In unit square, side $s=\sec(15^\circ)$ and area $A=2\sqrt{3}-3$.}

---

Another relevant paper from the same period:

Fekete, Sandor P. "Finding all anchored squares in a convex polygon in subquadratic time." (1992). Download author's PDF.

Not an answer, just two remarks.

(1) Literature. There is an old paper that finds a largest inscribed equilateral triangle, and a largest inscribed square, but these results do not straightforwardly generalize:

DePano, A., Yan Ke, and J. O’Rourke. "Finding largest inscribed equilateral triangles and squares." In Proc. 25th Allerton Conf. Commun. Control Comput, pp. 869-878. 1987.

Unfortunately, I no longer have access to my own article. :-/ My recollection is that we achieved $O(n^3)$ time for equilateral triangles and $O(n^2)$ time for squares, for a convex polygon of $n$ vertices.

---

     
^{See [Mathworld](http://mathworld.wolfram.com/EquilateralTriangle.html): In unit square, side $s=\sec(15^\circ)$ and area $A=2\sqrt{3}-3$.}

---

Another relevant paper from the same period:

Fekete, Sandor P. "Finding all anchored squares in a convex polygon in subquadratic time." (1992). Download author's PDF.

(2) Question 3: "does $A(n)$ have exactly one local maximum (which is also the global max) for any $C$?"

Let $C$ be an equilateral triangle. Then the global max is $A(3)$, but $A(6)$ is a local max:

---

          [![Hex_in_Tri][1]][1]
          ^{$A(3) > A(4) < A(6) > A(12)$.}

---

So perhaps you should exclude $C$ being a regular polygon itself.