I think looking at Bredon's Sheaf theory book (page $179$ chapter $3$) would be helpful.

It defines singular cohomology of a topological space $X$ with coefficients from **a sheaf of abelian groups** $\mathcal{A}$ by $H^k_{Sing}(X,\mathcal{A})$ which boils down to (I did not check in detail but I am very much sure) usual definition of singular cohomology of a space $X$ with values in abelian group $A$ when $\mathcal{A}$ is just the sheaf associated to constant presheaf $U\mapsto A$ for each $U$ open in $X$ i.e.,
$$H^k_{Sing}(X,\mathcal{A})\cong H^k_{Sing}(X,A)$$
where right hand side is usual singular cohomology with coefficients from $A$ and $\mathcal{A}$ is the sheaf associated to constant presheaf defined as $U\mapsto A$ for each $U$ open in $X$.

This way of seeing gives **less surprise** and **more clarity** for me.

It might be **surprising** to see that

On a paracompact space $X$, there is an isomorphism $H^k_{Sing}(X,A)\cong H^k_{Sheaf}(X,\underline{A}_X)$ where $\underline{A}_X$ is the sheaf associated to constant presheaf defined as $U\mapsto A $ for all $U$ open in $X$.

It is definitely little **less surprising** and **more natural** to see that

On a paracompact space $X$, there is an isomorphism $H^k_{Sing}(X,\mathcal{A})\cong H^k_{Sheaf}(X, \mathcal{A})$ for any sheaf of abelian groups $\mathcal{A}$ on $X$.

This proof is in Bredon's book. I do not prove that result but I mentioned just to give some motivation.

To define sheaf cohomology $H^k_{Sing}(X,\underline{A}_X)$ we need to look for a injective resolution of $\underline{A}_X$ and apply Global section functor to get Sheaf cohomology $H^k_{Sing}(X,\underline{A}_X)$.

Ramanan's Global calculus book (page $105$ lemma $3.1,3.2$) says

Let $0\rightarrow \mathcal{F}\rightarrow \mathcal{G}^\bullet$ be **an arbitrary** resolution of the $\mathcal{F}$. Suppose $H^i(X,\mathcal{G}^j)=0$ for each $i>0$ and $j\geq 0$. Then the complex $\mathcal{G}^\bullet$ has cohomologies canonically isomorphic to those of $\mathcal{F}$ i.e.,$H^k(\mathcal{G}^\bullet(X))$ is isomorphic to $H^k(X,\mathcal{F})$ for each $k>0$.

This result is relavent here because the idea is to produce a resolution of $\underline{A}_X$ that has the above property and we use that resolution to produce sheaf cohomology groups $H^k(X,\underline{A}_X)$.

Given an open set $U$ of $X$, consider singular $n$-cochains
inside $U$ i.e., $\alpha:\{\Delta_n\rightarrow U\}\rightarrow A$. Here, $\Delta_n$ is the standard$\alpha:\{\Delta_n\rightarrow U\}\rightarrow A$ $n$-simplex and $\Delta_n\rightarrow U$ is $n$-singular simplex in $U$ and $\alpha:\{\Delta_n\rightarrow U\}\rightarrow A$ is $n$-singular cochain in $U$ with values in $A$. The collection of all such $\alpha$ is denoted by $S^k(U)$. This is an abelian group. $\alpha:\{\Delta_n\rightarrow U\}\rightarrow A$

The assignment $U\rightarrow S^k(U)$ gives a presheaf $S^k$ of abelian groups on $X$. The sheaf associated to this presheaf is denoted by $\widetilde{S^k}$ on $X$ for each $X$. We get a resolution
$$0\rightarrow \mathcal{F}\rightarrow \widetilde{S^0}\rightarrow \widetilde{S^1}\rightarrow\cdots$$
In case when $X$ is **paracompact** the sheaf $\widetilde{S^k}$ is flabby sheaf (also called as flasque sheaf) for each $k$. For every flabby sheaf $\mathcal{G}$ we have $H^i(X,\mathcal{G})=0$ for each $i>0$. Thus, we can use this resolution $$0\rightarrow \widetilde{S^0}
\rightarrow \widetilde{S^1}\rightarrow \cdots$$
Taking global section functor we have
$$0\rightarrow \widetilde{S^0}(X)
\rightarrow \widetilde{S^1}(X)\rightarrow \cdots$$
As $\widetilde{S^k}(X)$ is related to presheaf of singular $n$-cochains on $X$, it is only expected that this cochain complex has same cohomology groups as that of
$$0\rightarrow \mathcal{C}^0(X)\rightarrow \mathcal{C}^1(X)\rightarrow $$
where $\mathcal{C}^i(X)$ is just the singular $n$-cochains on $X$ with values in $A$. Cohomology groups of this cochain complex is precisely the singular cohomology groups $H^k(X,A)$. Thus, sheaf cohomology groups $H^k(X,\underline{A}_X)$ are isomorphic to singular cohomology groups $H^k(X,A)$.

As the sheaf cohomology $H^k(X,\underline{A}_X)$ agrees with Cech cohomology $H^k(\mathcal{U},\underline{A}_X)$ when we have good cover $\mathcal{U}$. As $X$ is **locally contractible** that cover that we choose from a line bundle can be choosen to be good cover. In that case, we have $H^k (\mathcal{U},\underline{A}_X)$ is canonically isomorphic to $H^k (\mathcal{U},\underline{A}_X)$.

Thus, comparing above two results, we see that
$$H^k_{\text{Cech}}(\mathcal{U},\underline{A}_X)\cong H^k_{\text{Sheaf}}(X,\underline{A}_X)\cong H^k_{\text{Sing}}(X,A)$$

Sheaf theory, chapter III. $\endgroup$a sheaf of groups $\mathcal{A}$.. it is more complicated than that of coefficients to be a group... But, as you said, it is only natural to use this notation as in case of. Sheaf cohomology we are allowing coefficients from a a sheaf.. so, in singular cohomology also we should allow coefficients from a sheaf... it looks reasonable to go for this generality but it is complicated as of now for me... I will check it again and see if I understand @abx $\endgroup$2more comments