Let the vertices of the triangle be $A$, $B$, and $C$, which we also use for the angle measures, opposite the sides of lengths $a$, $b$ and $c$ respectively.

Suppose we know that in the ideal configuration, a horizontal line cuts the triangle at $A$, and a vertical line cuts the triangle at $B$. Let $\theta$ be the angle between the horizontal line and side $AB$.

Then the measure of the possible set of horiztonal lines is $c \sin(\theta)+b \sin(A - \theta)$, and the
expected length of the side uncut by horizontal lines is
$$h=\frac{bc \sin(\theta)+bc \sin(A - \theta)}{c \sin(\theta)+b \sin(A - \theta)}$$

Similarly the measure of the possible set of vertical lines is $c\sin(\pi/2-\theta)+a \sin(B-(\pi/2-\theta))$, or $c\cos(\theta)-a\cos(B+\theta)$, and the expected length of the side uncut by horizontal lines is
$$v=\frac{ac\cos(\theta)-ac\cos(B+\theta)}{c\cos(\theta)-a\cos(B+\theta)}$$

So the problem asks to maximize $\ell=(h+v)/2$ over all possible $\theta$. *[Update: that was for the version of the problem
with equal likelihood of horizontal and vertical lines. On one interpretation of the current version, $\ell$ might instead be the sum of the numerators of the above $h$ and $v$, divided by the sum of their denominators. In any case:]* The maximum has a closed-form expression, but it would be unilluminating.

So the full procedure here is:

- solve for $A$, $B$, and $C$.
- find $\theta$ and $\ell$ assuming that the horizontal line cuts $A$ and the vertical line cuts $B$ as above
- find $\theta$ and $\ell$ in the other eight cases for which vertex is cut by the horizontal line and which vertex is cut by the vertical line
- see which of these nine possibilities for $\theta$ and $\ell$ actually represent geometrically sensible arrangements
- choose the geometrically sensible possibility with highest $\ell$.

function of a, b and c)" and "over all orientations of T on the plane andall possible choices of a, b and c" are incompatible. I assume the latter is just a typo (otherwise the story is trivial: put $a=b=c=0$), right? $\endgroup$ – fedja Jul 30 '20 at 15:47whyyou're trying to maximize this quantity may help with giving more guidance and avoiding XY problems. $\endgroup$ – Steven Stadnicki Jul 30 '20 at 16:466more comments