$\newcommand{\diag}{{\rm diag}}
\newcommand{\sH}{{\mathcal H}}
\newcommand{\R}{{\mathbb R}}
\newcommand{\HH}{\sf H}
\newcommand{\V}{{\sf V}}
\newcommand{\B}{{\sf B}}
\newcommand{\C}{{\Bbb C}}
$No, the kernel $\ker \big[H^1(\R,\mu)\to H^1(\R,G)\big]$ does not have to be a subgroup of the abelian group $H^1(\R,\mu)$.
Indeed, let $G={\rm SU}(2,1)$, the special unitary group of the Hermitian form
$$\sH(z_1,z_2,z_3)=z_1\bar z_1+z_2\bar z_2-z_3\bar z_3$$
corresponding to the Hermitian matrix $\diag(1,1,-1)$.
Let $T\subset G$ denote the diagonal maximal torus, and set
$$\mu=T_2:=\{1,a,b,ab\}$$
where
\begin{align*}
1&=\diag(1,1,1),\\
a&=\diag(-1,1,-1),\\
b&=\diag(1,-1,-1), \\
ab&=\diag(-1,-1,1).
\end{align*}
Then
$$H^1(\R,\mu)=Z^1(\R,\mu)=\mu(\R)_2=\mu(\R)=\mu({\mathbb C})=\{1,a,b,ab\}.$$
We use the 1-cocycles $a,b,ab$ to twist the Hermitian form $\sH$.
We obtain the twisted Hermitian forms
\begin{align*}
_a\sH &=-z_1\bar z_1+z_2\bar z_2+z_3\bar z_3\,,\\
_b\sH &=z_1\bar z_1-z_2\bar z_2+z_3\bar z_3\,,\\
_{ab}\sH &= -z_1\bar z_1-z_2\bar z_2-z_3\bar z_3\,.
\end{align*}
We see that $_a\sH$ and $_b\sH$ are equivalent to $\sH$, whereas $_{ab}\sH$ is not.
Thus
$$\ker\big[H^1(\R,\mu)\to H^1(\R,G)\big]=\{1,a,b\},$$
which is not a subgroup of the group $H^1(\R,\mu)=\{1,a,b,ab\}$.
Edit. At the request of OP, I compute twisting of the Hermitian form in one variable
$$\sH(z,w)=\lambda \bar z w\quad\ \text{with}\ \, \lambda\in\R^\times$$
by the cocycle $-1$.
Let
$$ \HH\colon V\times V\to\C $$
be a Hermitian map, where $\V=\C^1$.
Then in coordinates
$$\sH(z,w)=\lambda\bar z w$$
where $\lambda\in\R^\times$.
We work over $\R$, so $\HH$ is given by two bilinear maps
$$ \B_1\colon \V\times \V\to\R,\quad\ \B_2\colon \V\times\V\to\R$$
(the real and the imaginary parts of $\HH$).
In coordinates they are given as
\begin{align*}
&B_1\big((z_1,z_2),(w_1,w_2)\big)=\lambda(z_1w_1+z_2w_2),\\
&B_2\big((z_1,z_2),(w_1,w_2)\big)=\lambda(z_1w_2-z_2w_1).
\end{align*}
We regard $\V$ as a two-dimensional $\R$-space with basis $e_1,e_2$.
We consider $\V_\C$ with basis $e_1,e_2$ and the standard complex conjugation
$$\sigma(x_1e_1+x_2e_2)=\bar x_1e_1+\bar x_2e_2.$$
We twist the pair $(\V,\HH)$ by the cocycle $-1$.
That is, we consider the new complex conjugation $\tau=-\sigma$,
$$\tau(x_1e_1+x_2e_2)=-\bar x_1e_1-\bar x_2e_2.$$
Then the fixed point set in $\V_\C$ of the twisted complex conjugation $\tau$
is the real vector space $\V'$ with basis
$$e_1'=ie_2,\ \,e_2'=ie_2.$$
In this new basis, our vector
$$z=(z_1,z_2)=z_1e_1+z_2e_2$$
is written as
$$z=z'_1e'_1+z'_2e'_2\quad\ \text{with}\ \, z_1'=-iz_1,\ z_2'=-iz_2,$$
and our bilinear maps $\B_1, \B_2$ are written in coordinates as
\begin{align*}
&B'_1\big((z'_1,z'_2),(w'_1,w'_2)\big)=-\lambda(z'_1w'_1+z'_2w'_2),\\
&B'_2\big((z'_1,z'_2),(w'_1,w'_2)\big)=-\lambda(z'_1w'_2-z'_2w'_1).
\end{align*}
This means that twisting by the cocycle $-1$ sends
our bilinear forms $(B_1,B_2)$ to $(-B_1,-B_2)$, and it sends
our Hermitian form $\sH$ to $-\sH$ (as expected!).
