Happened to have this slide in [one of my lectures](http://www.math.ntnu.no/~stacey/documents/tma4145.lectures.trans.2009-09-22.pdf).  The green circle and green dot correspond to the Euclidean norm, the purple square and purple line to the sup norm.  With the Euclidean norm, the green dot is _the_ closest point in the square to the blue dot; with the sup norm, the purple _line_ is the set of closest points in the square to the blue dot.

![closest point](https://i.sstatic.net/7zAR4.png)