Such a matrix defines an *unbounded* operator $\mathcal A$ over the Hilbert space $\ell^2(\mathbb N)$, with domain $D(\mathcal A)$, formed of vectors $x$ such that $\mathcal Ax\in H$. It may happen of course that $\mathcal A$ be a bounded operator. The property $\overline{a_{ij}}=a_{ji}$ tells you that $\langle\mathcal Ax,y\rangle=\langle x,\mathcal Ay\rangle$ whenever $x,y\in D(\mathcal A)$.

For an unbounded operator $\mathcal A$, we define the domain of the adjoint operator $\mathcal A^*$ to be the set of $y\in H$ such that the linear form $x\mapsto\langle\mathcal Ax,y\rangle$ is bounded.

Definition. The operator $\mathcal A$ is *self-adjoint* if $\langle\mathcal Ax,y\rangle=\langle x,\mathcal Ay\rangle$ whenever $x,y\in D(\mathcal A)$, and in addition $D(\mathcal A^*)=D(\mathcal A)$.

It turns out that the symmetry property does not imply the equality of the domains.
The theory of unitary diagonalisation of Hermitian matrices basically extends naturally to self-adjoint operators. See the book by Reed and Simon.