@Robert, you are right, it is not unique. Then how do you get the equality $\bigl(v\ \cdot_F\ v\bigr)\bigl(v\ \cdot_G\ w\bigr) = \bigl(v\ \cdot_G\ v\bigr)\bigl(v\ \cdot_F\ w\bigr).$

So in other words, give a line $L$ and a quadratic form $F$, there exists a unique linear map $s:\mathbf{R}^n\rightarrow\RR^{n}$ such that $s(w)=-w$ for all $w\in L$, $s^2=1$ and $s$ preserves $F$-length. So this uniqueness result is important in your argument.

Also a key observation in your argument is that the reflection $\rho_v^F$ only depends on the $\mathbf{C}$-line spanned by $v$, namely $\mathbf{C}v$. moreover there exists a unique line $L$ such that for all $w\in L$ one has that $\rho_v^F(w)=-w$ namely the line spanned by $v$. So using this obervation with the assumption that $O(F)=O(G)$ one may deduce that $\rho_v^F=\rho_v^G$.

Very nice computational proof @Will, thanks a lot!

So I solved my confusion, so if one takes the one-parameter subgroup $\left(\begin{array}{cc} a & 0 \newline 0 & 1\end{array}\right)$ one might simply conjugate it by an element of the form $\left(\begin{array}{cc} 1 & b \newline 0 & 1\end{array}\right)$

But then I don't quite see how you can generate this group with algebraic one-parameter subgroups "generated" by diagonalizable elements...

So I guess that the "simplest" example of a non-reductive real algebraic that contains a semi-simple element is the set of invertible matrices of form $\left( \begin{array}{cc} a & b \newline 0 & 1 \end{array} \right) $

Thanks algori, I did not think of taking the exponential of diagonalizable element! So for this density result on diagonalizable elements, are you assuming that $G$ is reductive?

So my maximal $\mathbf{R}$-tori will look like $(\mathbf{R}^{\times})^n\times (S^1)^m$ where $S^1$ stands for the unit circle. May be also I should have restricted my question to real algebraic reductive group

By an $\mathbf{R}$-torus I mean the $\mathbf{R}$-valued points of an algebraic group $H$ defined over $\mathbf{R}$ such that $H\otimes_{\mathbf{R}}\mathbf{C}\simeq (\mathbb{G}_m/\mathbf{C})^n$. So for example, with a rank $1$ torus (case $n=1$), if $T$ is split then $H(\mathbf{R})\simeq \mathbf{R}^{\times}$ and if it is non-split (and non-empty) it is isomorphic to $SO(2)$.