I think you are describing Siegel's construction for counting the representations of a number by a genus of quadratic forms. For positive forms, you count the number of representations by each class in the genus, but divide each one by the number of integer automorphs (isometries) of the particular form. In the end, you divide by the mass of the genus, the sum of the reciprocals of the automorph counts. For indefinite forms, all of that goes sideways, instead you count the orbits that you are describing, and this is finite, and not dependent on dimension. I will see if I can find a good description...