This is related to something which has always annoyed me: some authors define, say, the group algebra $k[G]$ of a group $G$ as the set of functions $G \to k$ with finite support, equipped with convolution. In my opinion this is a bad definition, because it misleads you as to the functoriality of this construction with respect to $G$. Namely, functions pull back but the group algebra pushes forward: if $f : G \to H$ is a group homomorphism, it induces an algebra homomorphism $k[G] \to k[H]$ which is annoying to write down in terms of functions but, in terms of the free vector space on $G$, trivial to write down as

$$\sum k_g g \mapsto \sum k_g f(g).$$

One way to describe what is going on is that the free vector space on a set $X$ consists of "distributions with finite support" on $X$. Functions pull back but distributions push forward, and this suggests the correct functoriality of taking the free vector space, which is covariant rather than contravariant. Moreover the free vector space functor is monoidal: it converts direct products into tensor products, which is an abstract explanation of why it converts groups into algebras.

The relevance of this discussion to continuous convolution is that if $G$ is, say, a locally compact Hausdorff group equipped with Haar measure, and we want to define some sort of group algebra on $G$, whatever that construction means it should be covariantly functorial, which as above naturally suggests we ought to consider distributions rather than functions; if distributions are defined as dual to a topological vector space of functions which pull back then they will naturally push forward. Then the identity will be the Dirac delta supported at the identity, exactly as in the discrete case; functoriality with respect to the inclusion $1 \to G$ of the identity forces this.

The assignment $X \mapsto C_c(X)$ of continuous functions with compact support, on the other hand, is not functorial at all, so it's not a good replacement for the free vector space functor.

This is related to something which has always annoyed me: some authors define, say, the group algebra $k[G]$ of a group $G$ as the set of functions $G \to k$ with finite support, equipped with convolution. In my opinion this is a bad definition, because it misleads you as to the functoriality of this construction with respect to $G$. Namely, functions pull back but the group algebra pushes forward: if $f : G \to H$ is a group homomorphism, it induces an algebra homomorphism $k[G] \to k[H]$ which is annoying to write down in terms of functions but, in terms of the free vector space on $G$, trivial to write down as

$$\sum k_g g \mapsto \sum k_g f(g).$$

One way to describe what is going on is that the free vector space on a set $X$ consists of "distributions with finite support" on $X$. Functions pull back but distributions push forward, and this suggests the correct functoriality of taking the free vector space, which is covariant rather than contravariant. Moreover the free vector space functor is monoidal: it converts direct products into tensor products, which is an abstract explanation of why it converts groups into algebras.

The relevance of this discussion to continuous convolution is that if $G$ is, say, a locally compact Hausdorff group, and we want to define some sort of group algebra of $G$, whatever that construction means it should be covariantly functorial, which as above naturally suggests we ought to consider distributions rather than functions; if distributions are defined as dual to a topological vector space of functions which pull back then they will naturally push forward. Then the identity will be the Dirac delta supported at the identity, exactly as in the discrete case; functoriality with respect to the inclusion $1 \to G$ of the identity forces this.

The assignment $X \mapsto C_c(X)$ of continuous functions with compact support, on the other hand, is not functorial at all, so it's not a good replacement for the free vector space functor. This isn't about completeness, really.