If that's the case, then try Dacorogna's Direct Method in the Calculus of Variations.

By background, you meant Sobolev spaces?

I think any calculus of variations book will have this result, try Dacorogna's Introduction to Calculus of Variations for example.

Why is $F_\varepsilon$ not equicoercive? I might have misunderstood the definition of equicoercive. Isn't equicoercive equivalent to showing that any uniformly bounded sublevel sets has a convergent subsequence?

@AnthonyCarapetis Great, I will check it out tomorrow when I have the time. Thank you so very much!

@AnthonyCarapetis Chapter 8 turns out to be fixed point theorems, unless I am looking at the wrong book. The book is "Second Order Parabolic DEs"?

@AnthonyCarapetis You are absolutely right. It was supposed to be Neumann instead of Dirichlet boundary conditions. I have edited the question.

@AnthonyCarapetis It should be $u(x,t)$, unless you meant otherwise?

@MichaelRenardy My intention is to learn how to dealt with the integral side constraint that is only defined on parts of the boundary. I agree that (1) and (4) are both redundant questions. The Dirichlet energy functional is simply a toy problem, my actual problem consists of (quadratic) functional of two variables. As I mentioned, the only way I know how to deal with integral constraint is by showing that they are weakly continuous, but I'd be happy to hear your thoughts.