Here is a different, perhaps more elementary example.
Consider the Grassmannian $\mathrm{Gr}(2,4)$ of lines in $\mathbf{P}^3$, and let $\mathscr{Q}$ be the universal quotient bundle. The line bundle $\mathrm{det}(\mathscr{Q})$ generates the Picard group of $\mathrm{Gr}(2,4)$. If $[\mathscr{Q}]$ were in the subgroup of $\mathrm{K}^0$ generated by line bundles, then we would have $[\mathscr{Q}]=\sum_i n_i [\mathrm{det}(\mathscr{Q})^{\otimes r_{i}}]$ for some integers $n_i, r_i$. Taking Chern classes implies $$c_2(\mathscr{Q})= m c_1(\mathscr{Q})^2$$ for some integer $m$, which cannot be true. Indeed, fix a point $p$ and a line $L$ in $\mathbf{P}^3$; the Chern class $c_1(\mathscr{Q})$ is geometrically represented by the Schubert cycle $\Sigma_1(L)=\{L' \ | \ L\cap L'\neq\emptyset\}$, while $c_2(\mathscr{Q})$ is represented by $\Sigma_2(p)=\{L' \ | \ p\in L'\}$. Then $$1=\int_{\mathrm{Gr}(2,4)} c_2(\mathscr{Q})^2\neq m^2\int_{\mathrm{Gr}(2,4)} c_1(\mathscr{Q})^4=2m^2.$$ The integral on the left is the number of lines through two distinct points in $\mathbf{P}^3$, while the integral on the right is the number of lines meeting four given (general) lines in $\mathbf{P}^3$.
I think only few varieties $\mathrm{X}$ will have the property which you want. It is certainly true for curves: if you have a locally free sheaf $\mathscr{E}$ on a curve, then you can find an exact sequence of locally free sheaves $$0\rightarrow\mathscr{E}'\rightarrow\mathscr{E}\rightarrow\mathscr{E}''\rightarrow 0,$$ where the ranks of $\mathscr{E}'$ and $\mathscr{E}''$ are strictly smaller than $\mathscr{E}$ (unless, of course, $\mathscr{E}$ is already of rank $1$). The same property holds if $\mathscr{E}$ is a locally free sheaf of rank $>2$ on a surface. Indeed, after twisting with an invertible sheaf, we may assume that $\mathscr{E}$ is globally generated; by a standard result (e.g. Exercise 8.2 in Chapter 2 of Hartshorne) we can then find a nonzero section $s:\mathscr{O}\rightarrow\mathscr{E}$ whose cokernel is locally free. The K-theory of surfaces can thus be generated by locally free sheaves of rank $1$ and $2$, and analogous statements hold in higher dimension.