If $n$ is odd, then the answer is $\lceil \binom{n}{2}/4 \rceil$, which is best possible.  This follows from two special cases of a more general conjecture by Alspach.  

For our purposes, we can use a [theorem](https://www.researchgate.net/publication/265366840_On_Alspach's_conjecture) of Heinrich, Horák, and Rosa which says that if $n$ is odd and $a,b,c$ are such that $3a+4b+6c=\binom{n}{2}$, then $E(K_n)$ can be partitioned into $a$ $3$-cycles, $b$ $4$-cycles, and $c$ $6$-cycles. They also proved the same theorem with $3,4,6$ replaced by $3,4,5$. 

Thus, if $n$ is odd, it is always possible to decompose $E(K_n)$ into $4$-cycles and possibly one extra cycle that is a $3$-cycle, a $5$-cycle or a $6$-cycle.  The edge set of the extra cycle can obviously be covered with two $4$-cycles of $K_n$, so we are done. 

If $n$ is even, then each vertex has odd degree.  Since every $4$-cycle passes through each vertex an even number of times, then for each vertex $v$, there will be at least one edge incident to $v$ that is covered twice.  Thus, there will be at least $n/2$ edges that are covered twice.  Thus, every covering of $E(K_n)$ by $4$-cycles has size at least $\binom{n}{2}/4+n/8$. We prove that this bound can actually be achieved using the same special case of Alspach's conjecture.  

Namely, for $n$ even, the same two decomposition results hold except with $K_n$ replaced by $K_n$ minus a perfect matching $M$, and $\binom{n}{2}$ replaced with $\frac{n(n-2)}{2}$. Since $\frac{n(n-2)}{2}$ is divisible by $4$, it follows that the edges of $K_n-M$ can be decomposed into $4$-cycles. By then covering pairs of edges $M$ with $4$-cycles we get a covering of size $\lceil \binom{n}{2}/4+n/8 \rceil$.