Thank you. What if I add the assumption that the boundary is analytic or piecewice analytic? I know I can extend the exterior potential analytically across the boundary, but I need to find some singular point in the extension.

Thanks for your concern. The "singular analytic" in my mind is the function that is analytic in the domain except several points and those points are unremovable, i.e you can not define the value at those points to make the function to be analytic in the whole domain. For exmaple, in the case of a ball the center is an unremovable singular point of analytic function $\frac{1}{|x-a|^{n-2}}$, which is the exterior potential of the ball.