The presentation $\mathbb{S}^{2n-1} = \mathit{U}_n/\mathit{U}_{n-1}$ is tantamount to considering $\mathbb{S}^{2n-1}$ as the unit sphere in $\mathbb{C}^n$ (as $\mathit{U}_n$ is the group of $\mathbb{C}$-linear maps of $\mathbb{C}^n$, acting, say, from the right, preserving the Hermitian inner product and $\mathit{U}_{n-1}$ is the stabilizer of a given vector). Now $\mathbb{C}^n$ has the structure of a $\mathbb{C}$-vector space, so you can quotient out by complex numbers of unit norm, $\mathit{U}_1$, getting, as you say, $\mathbb{P}^{n-1}(\mathbb{C})$ as $\mathbb{S}^{2n-1}/\mathit{U}_1 = \mathit{U}_n/(\mathit{U}_{n-1}\times \mathit{U}_1)$. In other words, the $(2n-1)$-sphere is fibered in circles with base $\mathbb{P}^{n-1}(\mathbb{C})$ (for $n=2$, noting that $\mathit{SU}_2$ is isomorphic to $\mathit{Spin}_3$, this is the classical Hopf fibration $\mathbb{S}^1 \hookrightarrow \mathbb{S}^3 \to \mathbb{S}^2$).
The analogous description of $\mathbb{S}^{4n-1} = \mathit{Sp}_n/\mathit{Sp}_{n-1}$ is to consider $\mathbb{S}^{4n-1}$ as the unit sphere in the space $\mathbb{H}^n$ of quaternionic vectors with the standard Hermitian inner product, so that $\mathit{Sp}_n$ is the group of $\mathbb{H}$-linear maps (acting on the right, say, and where $\mathbb{H}$ acts on the left) preserving the product in question and $\mathit{Sp}_{2n-1}$ is the point stabilizer. This time we can quotient out by quaternions of unit norm, $\mathit{Sp}_1$, and this describes $\mathbb{P}^{n-1}(\mathbb{H})$ as $\mathbb{S}^{4n-1}/\mathit{Sp}_1 = \mathit{Sp}_n/(\mathit{Sp}_{n-1}\times \mathit{Sp}_1)$. In other words, the $(4n-1)$-sphere is fibered in $3$-spheres with base $\mathbb{P}^{n-1}(\mathbb{H})$ (for $n=2$, noting that $\mathit{Sp}_2$ is isomorphic to $\mathit{Spin}_5$, and $\mathit{Sp}_1 \times \mathit{Sp}_1$ to $\mathit{Spin}_3 \times \mathit{Spin}_3 = \mathit{Spin}_4$, this is the Hopf fibration $\mathbb{S}^3 \hookrightarrow \mathbb{S}^7 \to \mathbb{S}^4$).
(Of course you can also quotient $\mathbb{S}^{4n-1}$ by $\mathit{U}_1$ if you want, but I don't see how that can be a question since $\mathbb{S}^{4n-1}$ is already a $\mathbb{S}^{2m-1}$ for $m=2n$ so you already have the answer.)
And before you ask, there is also a Hopf fibration $\mathbb{S}^7 \hookrightarrow \mathbb{S}^{15} \to \mathbb{S}^8$ which makes it look like $\mathbb{S}^8 = \mathbb{P}^1(\mathbb{O})$ should be the quotient of an action of "the group of octonions of unit norm" on the unit sphere in $\mathbb{O}^2$ but, alas, there is no such thing as "the group of octonions of unit norm", and while it makes some sense to think of $\mathit{Spin}_9$ as the "octonionic $U_2$" in the description of $\mathbb{P}^1(\mathbb{O}) = \mathbb{S}^8$ as $\mathit{Spin}_9/\mathit{Spin}_8$, there is no way to decompose $\mathit{Spin}_8$ as the product of two "octonionic $U_1$" factors. Furthermore, while $\mathbb{P}^2(\mathbb{O}) = F_4/\mathit{Spin}_9$ can be defined (where $F_4$ can be thought of as some kind of "octonionic $U_3$"), there does not exist a fibration of $\mathbb{S}^{15}$ with base $\mathbb{P}^2(\mathbb{O})$ and fiber $\mathbb{S}^7$, so the analogies stop there.