Skip to main content
4 of 4
added 71 characters in body
Gjergji Zaimi
  • 85.6k
  • 4
  • 236
  • 402

I believe your question (and many other variations) are answered here. Also see this survey by I. Barany about extremal problems for convex lattice polytopes.

Actually the problem for the convex polygon inside the square was considered earlier than that. See here for an article that seems to be written more in the lines of your question. The result is that $$K(n)\sim 2\pi \left(\frac{n}{12}\right)^{2/3}.$$

Gjergji Zaimi
  • 85.6k
  • 4
  • 236
  • 402