To me, Hausdorff metric is an unaccustomed way of making such a space of spaces. I think I don't trust it because fixing a homotopy type gives you a set that is neither closed nor open in general.

But yes I believe the picture is that some kind of "space of spaces of homotopy type $X$" is closely related to $A(X)$.

Let's start with smooth manifolds, but of codimension zero. For a fixed $n$ and a finite complex $K\subset \mathbb R^n$, let $M_n(K)$ be the space of smooth compact $n$-manifolds $N\subset \mathbb R^n$ containing $K$ in the interior as a deformation retract. (Let's say, the simplicial set where a $p$-simplex is a suitable thing in $\Delta^p\times \mathbb R^n$ such that the projection to $\Delta^p$ is a smooth fiber bundle.) You can map $M_n(K)\to M_{n+1}(K)$ by crossing with $[-1,1]$ (and doing something about corners), and you can consider the (homotopy) colimit over $n$. Using the classification of $h$-cobordisms you can work out that the set of components is the Whitehead group of $K$. The loopspace of one component is the smooth stable pseudoisotopy space of $K$. To get the idea, think of the case when $K$ is a point: the space $M_n(K)$ is then, after you discard extraneous components corresponding to cases where the boundary is not simply connected -- which were going to go away anyway upon stabilizing over $n$ -- your quotient of {embeddings $D^n\to\mathbb R^n$}~$O(n)$ by {diffeomorphisms $D^n\to D^n$}. It's also a kind of "space of all $h$-cobordisms on $S^{n-1}$, and thus a delooping of the (unstable) pseudoisotopy space of $S^{n-1}$.

When $K$ is more complicated than a point, it's important to distinguish between the space of all blah blah blah containing $K$ as a deformation retract and the space of all blah blah homotopy equivalent to $K$; they differ by the space of homotopy equivalences $K\to K$.

There is a similar story for the piecewise linear or topological case.

The piecewise linear manifold version of this construction can, I believe, be shown to be equivalent to a non-manifold construction more like what you asked about: some kind of "space of compact PL spaces in $\mathbb R^n$ containing a fixed $K$ as deformation retract".

Waldhausen tells us that the stable smooth construction above and the stable PL construction above are respectively (the underlying spaces of spectra which are) the fiber of a map from the suspension spectrum of $K\cup{point}$ to $A(K)$, and the fiber of a map from $A(*)\wedge (K\cup{point})$ to $A(K)$.