Consider the following alternative definition of finite reflection group:
Definition: A finite reflection group $\Gamma\subset\mathrm O(\Bbb R^d)$ is a finite group generated by orthogonal transformations $T\in\mathrm O(\Bbb R^d)$ with eigenvalues $\{-1^1,1^{d-1}\}$. (the exponents denote multiplicites)
This definition suggests the following generalization:
Definition: A finite $k$-reflection group $\Gamma\subset\mathrm O(\Bbb R^d)$ is a finite group generated by orthogonal transformations $T\in\mathrm O(\Bbb R^d)$ with eigenvalues $\{-1^k,1^{d-k}\}$.
In other words: instead of inverting a 1-dimensional subspace, each generator inverts a $k$-dimensional subspace and leaves the orthogonal complement fixed.
As there are root systems associated with finite reflection groups, one can defined analogous systems for $k$-reflection groups. The elements of these are not vectors, but $k$-dimensional subspaces which are invariant w.r.t. certain "generalized reflections".
Question: Have such objects been studied before? Does there exist a classification?
Some thoughts
Let $\Gamma$ is a $k$-reflection group generated by "reflections" $T_U,U\in\mathcal U$, where $\mathcal U$ is the associated "generalized root system" that contains $k$-dimensional linear subspaces, and $T_U$ has eigenspace $U$ to eigenvalue $-1$. Then $\Gamma'$ generated by $T_{U^\bot}=-T_U,U\in\mathcal U$ is a $(d-k)$-reflection group.
So, all $(d-1)$-reflection groups are already classified via the usual reflection groups. In particular, up to dimension three, all generalized reflection groups are classified in this way. The first interesting case are the 2-reflection groups in $\Bbb R^4$, which are probably related to complex reflection groups.