If $a$, $b$, $c$ and $d$ are the four sides of a quadrilateral, the problem is to show that $ab^2(bc)+bc^2(cd)+cd^2(da)+da^2(ab)\ge 0$. I've verified it to be true for quite a large number of values, but can't seem to come up with a proof for it. Does anyone have any ideas, perhaps some inequality results that can be applied to prove it?

1$\begingroup$ Where does this inequality come from? $\endgroup$ – Stefan Kohl Dec 5 '13 at 17:46

1$\begingroup$ Well, the inspiration for this inequality comes from an old IMO problem which asks to prove the following, for a triangle: $a^{2}b(a  b) + b^{2}c(b  c) +c^{2}a(c  a)\ge 0.$ It is relatively easier to prove the latter, by substitution. $\endgroup$ – Train Heartnet Dec 5 '13 at 18:49

$\begingroup$ It's a nice question, but not research mathematics, IMHO $\endgroup$ – Igor Rivin Jan 13 '15 at 16:55

$\begingroup$ @IgorRivin: What do you think are the benefits of closing a yearold answered question like this? $\endgroup$ – Stefan Kohl Jan 13 '15 at 17:58

1$\begingroup$ @StefanKohl I did not realize it was a year old... $\endgroup$ – Igor Rivin Jan 13 '15 at 19:40
Assuming the contrary, we get that the minimal value of $$ f(a,b,c,d)=ab^2(b−c)+bc^2(c−d)+cd^2(d−a)+da^2(a−b) $$ on the set $$ M=\{(a,b,c,d)\colon a,b,c,d\geq 0, a+b+c+d\leq 1, a+b+c\geq d,\dots\} $$ is negative. By the homogeneity, this value is achieved on $a+b+c+d=1$.
Later we will check that $f$ is nonnegative on the boundary faces $a=0$ and $a+b+c=d$. Thus the minimum is achieved at an interior point of the face defined by $a+b+c+d=1$. By the (Karush)KuhnTucker theorem, this point satisfies the system $$ \lambda=b^2(bc)cd^2+3da^22dab=\cdots $$ for some nonnegative $\lambda$ (the other equalities are obtained by the cyclic permutation of variables). In particular, this means that $$ b^3+3da^2\leq b^2c+cd^2+2dab, \;\dots, $$ which sums up to $$ \sum a^3+2\sum ab^2\leq \sum a^2b+2\sum abc $$ (the sums are also over four cyclic permutations). On the other hand, by AMGM we have $$ a^3+ab^2+bc^2+bc^2\geq 4abc, \quad a^3+ab^2\geq 2a^2b. $$ Summing up all these we get the converse; thus all these inequalities should come to equalities, and hence $a=b=c=d$; but $f(a,a,a,a)=0$. A contradiction.
It remains to check the inequality on the other boundary faces. If $a=0$ then the inequality looks like $bc^2(dc)\leq cd^3$; clearly we may assume that $d\geq c$, otherwise the inequality is trivial. Since $b\leq c+d$, we have $ bc^2(dc)\leq c^2(d^2c^2)\leq cd\cdot d^2$, as required.
Finally, if $a=b+c+d$ then we may substitute this into our function getting (here Maple was used...) the large sum with many positive terms (including $b^4+bc^3$) and only one negative term $b^2c^2$. But it is easy to see that $b^4b^2c^2+bc^3\geq 0$ for all nonnegative $b$ and $c$.
Let $T$ be the function in question: $$ T(a,b,c,d) = a b^2(bc) + b c^2 (cd) + c d^2 (da) + d a^2 (a  b). $$ We wish to show $T(a,b,c,d)\ge 0$ if $a,b,c,d$ are the sides of a quadrilateral. (Presumably, $a$ is the side opposite $c$ and $b$ is opposite $d$, but it actually doesn't matter to the proof.)
Terminology We introduce the following terminology: If $x_1,x_2,x_3,x_4$ are four real numbers (possibly negative), we say $x_1,x_2,x_3,x_4$ are "quadrilateral" if the following constraint holds: \begin{eqnarray} (*)\ \ \ \ x_1 + x_2 + x_3 + x_4 &\ge& 2 \max\{x_1,x_2,x_3,x_4\}. \end{eqnarray}
We also say that $x_1,x_2,x_3,x_4$ are "linear quadrilateral" if $(*)$ holds with equality.
Obviously, the sides $a,b,c,d$ of a quadrilateral are, well, quadrilateral. If the quadrilateral is degenerate so that its four vertices fall on a line, then its sides are linear quadrilateral.
Basic Idea The basic idea of the proof is to continuously "shrink" the sides $a,b,c,d$ of the quadrilateral by equal amounts $x$ until the quadrilateral collapses and all four of its vertices fall on a line. In other words, $(ax,bx,cx,dx)$ are linear quadrilateral. In step 3 below, we show this shrinking process decreases $T$, i.e., $T(ax,bx,cx,dx)$ is decreasing in $x\ge 0$ until $ax,bx,cx,dx$ are linear quadrilateral. The complexity comes when one realizes that during this shrinking process, one of the sides may collapse through a point and its length become negative (in which case it no longer makes sense to talk about $ax,bx,cx,dx$ being sides of a quadrilateral). Step 1 handles this "negative" case. Step 2 handles the more natural case in which no side becomes negative during the shrinking process.
Step 1 Suppose $a,b,c,d\ge 0$ are quadrilateral and one of $a,b,c,d$ vanishes. Then $T(a,b,c,d)\ge 0$.
Proof Assume without loss of generality that $a=0$. Then $$ T(0,b,c,d) = b c^3  b c^2 d + c d^3. $$ Observe \begin{eqnarray} c \ge d &\implies& b c^3  b c^2 d \ge 0 \implies T(0,b,c,d) \ge 0 \\ d \ge b,c &\implies& c d^3  b c^2 d \ge 0 \implies T(0,b,c,d) \ge 0. \end{eqnarray} The only other case not covered by these two conditions is $b> d > c$. In this case, we use use the fact that $a,b,c,d$ are quadrilateral to deduce $b\le c+d$. Since $cd<0$, \begin{eqnarray} T(0,b,c,d) &=& b c^2(cd) + c d^3 \\ &\ge& c^2 (c+d)(cd) + c d^3 \\ &=& c^4  c^2 d^2 + c d^3 \\ &=& c^4 + c d^2 (d  c) \\ &\ge& 0. \end{eqnarray} In any case, $T(0,b,c,d)\ge 0$.
Step 2 Suppose $a,b,c,d\ge 0$ are linear quadrilateral. Then $$ T(a,b,c,d)\ge 0. $$
Proof Without loss of generality, suppose $d=\max\{a,b,c,d\}$, so $d=a+b+c$.
By direct computation, \begin{eqnarray} T(a,b,c,a+b+c) &=& a^4  a^2 b^2 + a b^3 + a^3 c + a b^2 c + b^3 c + a^2 c^2 + 3 a b c^2 \\ & & \ + 2 b^2 c^2 + 2 a c^3 + 3 b c^3 + c^4. \end{eqnarray}
Luckily, the only summand that can possibly be negative is $a^2 b^2$. Observe \begin{eqnarray} a\ge b &\implies& a^4  a^2 b^2 \ge 0 \\ a\le b &\implies& a b^3  a^2 b^2 \ge 0. \end{eqnarray} Thus, $$ T(a,b,c,a+b+c) \ge a^4 + a b^3  a^2 b^2 \ge 0. $$
Step 3 Suppose $a,b,c,d$ are linear quadrilateral and not all equal. Suppose the sum of any two of $a,b,c,d$ is nonnegative. Then for $x\ge 0$, the mapping $$ x \mapsto T(a+x,b+x,c+x,d+x) $$ is strictly increasing in $x$.
Proof
Without loss of generality, let $d=\max\{a,b,c,d\}$. Since $a,b,c,d$ are linear quadrilateral, $$ d = a + b + c. $$
Direct computation shows $$ T(a+x,b+x,c+x,d+x) = T(a,b,c,d) + A x + B x^2 $$ where \begin{eqnarray} A &=& a^3  a^2 b + 2 a b^2 + b^3  2 a b c  b^2 c + 2 b c^2 + c^3 \\ & & \ + 2 a^2 d  2 a b d  2 a c d  2 b c d  c^2 d  a d^2 + 2 c d^2 + d^3. \end{eqnarray} and \begin{eqnarray} B &=& 2 a (a  b) + a (b  c) + 2 b (b  c) + b (c  d) \\ & & \ + 2 c (c  d) + (a  b) d + c (a + d) + 2 d (a + d) \\ &=& (ac)^2 + (bd)^2 + \frac{1}{2}(ab)^2 + \frac{1}{2}(ad)^2 + \frac{1}{2}(bc)^2 + \frac{1}{2}(cd)^2. \end{eqnarray} Clearly, $B>0$ because $a,b,c,d$ are not all the same.
Now substitute $d=a+b+c$ in the expression for $A$ and simplify: $$ A = 3 a^3 + 2 a b^2 + 2 b^3 + 3 a^2 c + 2 b^2 c + 3 a c^2 + 6 b c^2 + 3 c^3. $$
Suppose $a<0$. Since the sum of any two of $a,b,c,d$ is nonnegative, $b,c,d\ge a$. Observe \begin{eqnarray} A &=& 3 a^3 + 2 a b^2 + 2 b^3 + 3 a^2 c + 2 b^2 c + 3 a c^2 + 6 b c^2 + 3 c^3 \\ &=& (3 a^3 + 3a^2 c) + (2 a b^2 + 2 b^3) + (3 a c^2 + 3 c^3) + 2 b^2 c + 6 b c^2 \\ &\ge& 0. \end{eqnarray} (All the quantities in parentheses are nonnegative.)
Suppose $b<0$. Because the sum of any two of $a,b,c,d$ is nonnegative, $a,c,d\ge b$. We have \begin{eqnarray} A &=& 3 a^3 + 2 a b^2 + 2 b^3 + 3 a^2 c + 2 b^2 c + 3 a c^2 + 6 b c^2 + 3 c^3 \\ &=& (2 b^3 + 2 a b^2) + (6 b c^2 + 3 c^3 + 3 a c^2) + 3 a^3 + 3 a^2 c + 2 b^2 c \\ &\ge& 0. \end{eqnarray}
Finally, suppose $c<0$. Then $a,b,d\ge c$ and \begin{eqnarray} A &=& 3 a^3 + 2 a b^2 + 2 b^3 + 3 a^2 c + 2 b^2 c + 3 a c^2 + 6 b c^2 + 3 c^3 \\ &=& (3 a^2 c + 3 a^3) + (2 b^2 c + 2 a b^2)+ (3 c^3 + 3 a c^2) + 6 b c^2 + 2 b^3 \\ &\ge& 0. \end{eqnarray}
Since $B>0$ and $A\ge 0$, the result follows.
Step 4 Suppose $a,b,c,d\ge 0$ are quadrilateral and not all the same. Then $T(a,b,c,d) > 0$.
Proof Make the following definitions: \begin{eqnarray} x_0 &=& \frac{1}{2}(a+b+c+d  2\max\{a,b,c,d\}) \\ A &=& a  x_0 \\ B &=& b  x_0 \\ C &=& c  x_0 \\ D &=& d  x_0. \end{eqnarray}
It is easy to see $A,B,C,D$ are linear quadrilateral. Furthermore, the sum of any two of $A,B,C,D$ is nonnegative. For example, \begin{eqnarray} A+B &=& a + b  2 x_0 \\ &=& 2\max\{a,b,c,d\}  c  d \\ &\ge& 0. \end{eqnarray} All the other cases are just as easy.
Consider the function $f:[0,\infty)\to\mathbb{R}$ defined by $$ f(x) = T(A+x,B+x,C+x,D+x). $$ It follows from step 3 that $f$ is strictly increasing.
Since $a,b,c,d$ are quadrilateral, $x_0\ge 0$, so $T(a,b,c,d) = f(x_0) \ge f(0)$. If $A,B,C,D$ are all nonnegative, then $f(0)\ge 0$ by step 2 and we are done.
Let $m=\min\{a,b,c,d\}$ and suppose one of $A,B,C,D$ is negative, so $m < x_0$. Then $T(a,b,c,d) = f(x_0) > f(m)$. But by step 1, $f(m)\ge 0$ and we are done.

$\begingroup$ Seems to be directly plagiarized from this answer $\endgroup$ – user574848 Mar 24 at 11:29