If $n$ is odd, the answer is $\lceil \binom{n}{2}/4 \rceil$.
If $n$ is even, the answer is $\lceil \binom{n}{2}/4+n/8 \rceil$.
This follows from two special cases of a more general conjecture by Alspach.
For our purposes, we use a theorem of Heinrich, Horák, and Rosa which says that if $n$ is odd and $a,b,c$ are such that $3a+4b+6c=\binom{n}{2}$, then $E(K_n)$ can be partitioned into $a$ $3$-cycles, $b$ $4$-cycles, and $c$ $6$-cycles. Huang and Fu proved the same result with $(3,4,6)$ replaced by $(4,5)$.
Thus, if $n \geq 7$ is odd, it is always possible to decompose $E(K_n)$ into $4$-cycles and possibly one extra cycle that is a $3$-cycle, a $5$-cycle or a $6$-cycle. The edge set of the extra cycle can obviously be covered with two $4$-cycles of $K_n$, so we are done.
If $n$ is even, then each vertex has odd degree. Let $v$ be an arbitrary vertex. Since every $4$-cycle uses $0$ or $2$ edges incident to $v$, there will be at least one edge incident to $v$ that is covered twice. Thus, in total there will be at least $n/2$ edges that are covered twice. Thus, every covering of $E(K_n)$ by $4$-cycles has size at least $\binom{n}{2}/4+n/8$. We prove that this bound can actually be achieved.
Namely, for $n$ even, Heinrich, Horák, and Rosa's result holds except with $K_n$ replaced by $K_n$ minus a perfect matching $M$, and $\binom{n}{2}$ replaced with $\frac{n(n-2)}{2}$. For $n$ even, $\frac{n(n-2)}{2}$ is divisible by $4$. It follows that the edges of $K_n-M$ can be decomposed into $4$-cycles. By then covering pairs of edges of $M$ with $4$-cycles we get a covering of size $\lceil \binom{n}{2}/4+n/8 \rceil$.