Here is a partly baked idea: Hang a sheet--a light cloth--from the segment $(0,0,0)-(1,0,0)$ along the bottom of your square, and from the segment $(0,1,0)-(1,1,1)$ slanting above the top of your square, and let it sag under gravity below the $xy$-plane, pinned to these two segments. There are very nice cloth simulation algorithms implemented, e.g., in Blender below. It seems you could approximate the shape of the sheet by hanging a catenary between the points $(x,0,0)$ and $(x,1,x)$. By increasing the lengths of the catenaries as a function of $x$, it seems you should be able to make the surface concave throughout. Likely the catenaries could be replaced by parabolas.
![Cloth simulation][2]
**Addendum**. I defer to Pietro's more precise analysis, but just to hint toward the idea I suggested, here is a (substantially imperfect) rendition of the sagging sheet that (nearly) matches the boundary conditions:
![SheetSagging][3]