Slogan: The Hochschild homology is a (derived) categorification of the trace.
This means the identity at the end of John Pardon's answer is a categorification of the identity $\text{tr}(AB) = \text{tr}(BA)$.
To see this the idea is to think of bimodules as categorifications of linear maps. One way to think about this is in terms of the Eilenberg-Watts theorem, which identifies $(A, B)$-bimodules with cocontinuous functors $\text{Mod}(A) \to \text{Mod}(B)$: every such functor is tensoring with some $(A, B)$-bimodule. Composition is given by composition of cocontinuous functors, or tensoring bimodules.
A particularly nice special case is where $A = k^n, B = k^m$ for $k$ a field (and we consider bimodules over $k$): then $(A, B)$-bimodules are $m \times n$ "matrices" of vector spaces over $k$, and composition / tensor product is "matrix multiplication." This really makes the analogy to linear maps particularly explicit. For more details see this blog post.
Bimodules form part of a 2-category whose objects are rings (or more generally algebras over a base commutative ring), morphisms are bimodules, and 2-morphisms are morphisms of bimodules. This 2-category is symmetric monoidal with monoidal product given by the tensor product of rings, which satisfies a universal property analogous to the universal property of the tensor product of vector spaces: namely, functors $\text{Mod}(A) \times \text{Mod}(B) \to \text{Mod}(C)$ which are cocontinuous in each variable can be identified with functors $\text{Mod}(A \otimes B) \to \text{Mod}(C)$ (and hence with $(A \otimes B, C)$-bimodules).
Now, in any symmetric monoidal (higher) category one can define dualizable objects and traces of endomorphisms of dualizable objects; see this blog post for details and pictures, which among other things explains what taking traces has to do with circles. In this case
- every ring $A$ is dualizable with dual $A^{op}$,
- endomorphisms correspond to $(A, A)$-bimodules $M$, and
- the trace turns out to be the zeroth Hochschild homology $HH_0(A, M)$.
The full Hochschild homology is obtained by deriving this whole story, so that now morphisms are given by derived categories of bimodules and composition is given by the derived tensor product. All of this can be thought of as an elaboration of 1-dimensional topological field theory, where traces correspond to circles, and higher-dimensional topological field theory features generalizations of Hochschild homology involving more complicated manifolds.