Showing coercivity condition for an energy functional Consider the energy functional $e(\cdot)$ 
\begin{align*}
e(f,Q)&=\int_a^b \bigg\{f^4\bigg[1+\|\frac{d}{dr}Q\|^2+f^2\dot f^2\bigg]\bigg\} \,dr,
\end{align*}
over the space of 
\begin{equation*} 
{\mathcal E}:=\left\{ (Q, f) : 
\begin{array}{l}
Q \in W^{1,4}([a,b],{\bf SO}(n)),\\
Q(a)=Q(b)=I,\\
f \in W^{1,4}[a,b],\\
\dot f>0 \mbox{ ${\cal L}^1$-a.e. on $(a,b)$},\\
f(a)=a, f(b)=b. 
\end{array}
\right\}
\end{equation*} 
It is easy to show that, energy functional $e(\cdot)$ is coercive when $Q\in W^{1,2}$ and $f\in W^{1,2}$, in another words, there exists $d=d(n, a, b)>0$ such that
\begin{equation*} 
e(f, Q) \ge d ( \|Q\|^2_{W^{1,2}} + \|f\|^2_{W^{1,2}}).
\end{equation*}
Now my question is: 
The energy functional $e(\cdot)$ is coercive on the space $\mathcal E$ or not? In another words could we find $\gamma=\gamma(n, a, b)>0$ such that
\begin{equation*} 
e(f, Q) \ge \gamma ( \|Q\|^4_{W^{1,4}} + \|f\|^4_{W^{1,4}}).
\end{equation*}
for all $(f, Q)\in \mathcal E$.
 A: No. The reason is that
$$e(f,Q)=\int_a^b \bigg\{f^4\bigg[1+\|\frac{d}{dr}Q\|^2+f^2\dot f^2\bigg]\bigg\} \,dr,
$$
with $b\geq f\geq a$ (since $f$ is continuous and monotonous) gives 
$$
e(f,Q) \leq \left(1+\frac{b^4}{2}\right)\left(\|f\|^4_{W^{1,4}} + \|Q\|^2_{W^{1,2}}\right)
$$
So your inequality would imply that there exists $C_1,C_2>0$ such that for all $f,Q\in\mathcal{E}$
$$
C_1\|Q\|^2_{W^{1,2}} + C_2\|f\|^4_{W^{1,4}} \geq  \|Q\|^4_{W^{1,4}}.
$$
Now, fix $f=a+b\frac{t-a}{b-a}$, and it becomes for all $Q\in\mathcal{E}$
$$
C_3(\|Q\|^2_{W^{1,2}} + 1) \geq  \|Q\|^4_{W^{1,4}},
$$
if you cook-up a sequence $Q_n$  with $\|Q_n\|_{W^{1,2}}=1$ and  $\|Q_n\|_{W^{1,4}}>n$  you have a contradiction.  
Note that $W^{1,4}$ is not natural, since you can also obtain a bound from above of the form
$$
e(f,Q) \leq C(a,b)\left(\|f\|^2_{W^{1,2}} + \|Q\|^2_{W^{1,2}}\right).
$$
--- edited after the comment to the first part of the answer--
If you are looking for a minimizer, it is immediate to note that
$$
e(f,Q)\geq e(f,I)
$$
for any $f$ since it is a sum of squares, and $(f,I)\in \mathcal{E}$. Therefore you just want to solve 
$$
\min_{f\in \mathcal{X}} \int_a^b \left( f^4 +f^6 \left(\frac{df}{dt}\right)^2\right) \,dt,
$$ 
with $\mathcal{X}=\{f \in W^{1,2}(a,b)~:~f(a)=a,f(b)=b, f^\prime>0~a.e.\}$, a convex set. A calculus of variation problem in one dimension with a monotone integrand-> textbook question. The infimum will be in $\bar{\mathcal{X}}$ in general, and $f$ satisfies either the Euler-Lagrange equation, or is constant. 
To find a simple answer, let us assume that $0<a<b$. Then $f>0$ in $\mathcal{X}$, and if you set $g=f^4$, you find that your problem is also
$$
\min_{g\in \mathcal{Y}} \int_a^b \left( g + \frac{1}{16} \left(\frac{dg}{dt}\right)^2\right) \,dt,
$$ 
with $\mathcal{Y}=\{f \in W^{1,2}(a,b)~:~g(a)=a^4,g(b)=b^4, g^\prime>0~a.e.\}$, a convex, open set. 
Now, the Euler-Lagrange equation is simply $$1+\frac{1}{8}g^{\prime\prime}=0.$$ You can solve it by hand, and playing with it, you find for example that when $$a>
\frac{\sqrt{6}}{8}\frac{\left(3554+2(33)^{3/2}\right)^{1/3}}{\sqrt{7}\left(3554+2(33)^{3/2}\right)^{2/3}+(33)^{3/2}+116\left(3554+2(33)^{3/2}\right)^{1/3}+1777)},$$ 
the solution with $g(a)=a^4$ and $g(b)=b^4$ is strictly increasing and therefore is in $\mathcal{Y}$, whereas otherwise the positivity of the gradient constraint comes into play for suitable $b$s.    
A: that is a good idea !
i guess that your conjecture may has a counter-example !
see : http://www.stanford.edu/class/math220b/handouts/calcvar.pdf
that is because :
$e(f, Q) \ge d ( \|Q\|^2_{W^{1,2}} + \|f\|^2_{W^{1,2}})$$\Longrightarrow$$f^4(1+\vert{\nabla{Q}}\vert^2+f^2\vert{\nabla{f}}^2\vert)$$\ge$$d(Q^2+\vert{\nabla{Q}}\vert^2+f^2+\vert{\nabla{f}}^2\vert)$
we select $d=f^4$  to get :
$1+\vert{\nabla{Q}}\vert^2+f^2\vert{\nabla{f}}^2\vert$$\ge$$Q^2+\vert{\nabla{Q}}\vert^2+f^2+\vert{\nabla{f}}^2\vert$$\ge$$4Q\nabla{Q}+4f\nabla{f}$$\Longrightarrow$$f\nabla{f}(4-f\nabla{f})+Q\nabla{Q}(4-\frac{\nabla{Q}}{Q})\le1$
on the other hand ,
$f^4(1+\vert{\nabla{Q}}\vert^2+f^2\vert{\nabla{f}}^2\vert)$$-\gamma(Q^4+\nabla{Q}^4+f^4+\nabla{f}^4)\ge0$$\Longrightarrow$$1+\vert{\nabla{Q}}\vert^2+f^2\vert{\nabla{f}}^2\vert$$\ge$$4Q^2\nabla{Q}^2+4f^2\nabla{f}^2$$\Longrightarrow$$3f^2\nabla{f}^2+Q^2\nabla{Q}^2(4-\frac{1}{Q^2})\le1$
by comparing the two inequality above, we have :
$3f^2\nabla{f}^2+Q^2\nabla{Q}^2(4-\frac{1}{Q^2})\le$$f\nabla{f}(4-f\nabla{f})+Q\nabla{Q}(4-\frac{\nabla{Q}}{Q})$
$\Longrightarrow$$4f\nabla{f}(f\nabla{f}-1)+4Q\nabla{Q}(Q\nabla{Q}-1)\le0$
now, it is clear that your problem is related with the convexity and the concavity of the function you defined on $[a,b]$  !
A: The Euler-Lagrange Equations of the $e(f, Q)$ over the admissible space $\mathcal E$ arise as the following system 
\begin{align*}
\left \{ \begin {array}{ll}
(i)\ \ \frac{d}{dr} \bigg[ f^4 Q^t \frac{d}{dr} Q \bigg] ={\bf 0},\\
\\
(ii)\ \frac{d}{dr} \bigg[ f^2 \dot f \bigg] =
2f^3 +3f^5 \dot f^2 + 2f^3 |dQ|^2,
\end{array} \right.
\end{align*}
Now as the functional $e(\cdot, \cdot)$ is coercive on $\mathcal E$ when we choose the $f, Q$ in $W^{1,2}$ instead of $W^{1,4}$ and an application of direct methods shows the system has a solution. Well, could we claim this result in $\mathcal E$ by $f, Q$ in $W^{1,4}$? 
A: It is evident that your functional $e(f, Q)$ has minimizer in the space $W^{1,2}$ since you have coercivity in this space and an application of direct method of calculus of variation givee you the required result in $W^{1,2}$. So the system of Euler-Lagrange equation you wrote in below, has solution in $W^{1,2}$. Now from the first equation (equation (i) you can compute $\dot{Q}$ and $Q$, in fact we have 
$$ \dot{Q} = \frac{1}{f^4}Q C$$
in which $C$ is a (skew symmetric) constant matrix. therefore $|\dot{Q}|^4 < \infty$ and hence $Q \in W^{1,4}$ (in fact by this argument you can show that the solution $Q$ is smooth).  Similarly form the equation (ii) you can compute $f''$ in terms of $f$ and $f'$ and $|\dot{Q}|$ which shows that $f''$ exists (note that $f$ is bounded on [a,b]) so $f'$ is continuous and consequently for the solution $f\in W^{1,2}$ of the above system we have  $f\in W^{1,4}$.  In fact by the above argument one can show that the solution $f$ of the above system is also smooth (similar to the solution $Q$).  Therefore the solution (Q,f) which first was proved to be in the space $W^{1,2}$ is in the space $W^{1,4}$ and moreover by applying the system of equation (i) and (ii) similarly we could prove that this solution $(Q, f)$is a $C^{\infty}$ solution.    
