It sounds to me like what you're looking for is something like Cech (co)homology. The idea is that you can detect what kind of "paths" there are in a space by the combinatorics of which sets in open covers have nontrivial intersections. As a simple example, you can detect that a circle has a nontrivial loop by covering it with 3 open sets $U$, $V$, and $W$, such that any two of them intersect but $U\cap V\cap W$ is empty. More precisely, given any open cover, you can construct a simplicial complex which is the "nerve" of the open cover and has the "paths" that a space having that open cover "morally" should have. Of course, you shouldn't expect a single open cover to capture all of the information you're trying to capture about a space, so you have to take some sort of limit over all open covers of your space. Taking finer and finer open covers is like taking the "shortest possible steps" that you refer to.
As an example of how this might give you what you're looking for, Cech cohomology can't tell the difference between an ordinary circle and a loop built out of a topologist's sine curve, or an ordinary circle and a circle obtained by gluing together the two ends of a closed long line. It won't work for things like the hyperreals unless you restrict to some sort of internal open covers, because the hyperreals as a topological space are disconnected, and Cech cohomology detects topological connectedness (but not path-connectedness in the usual sense). And of course, if you're dealing with things like locally finite metric spaces which are discrete as spaces, there's no hope of saying anything interesting unless you endow the spaces with more structure than just a topology.
As a technical point, Cech cohomology is pretty well-behaved, but if you try to do the same thing with homology you run into problems because when you take a limit over all open covers you end up taking an inverse limit of homology groups, and taking inverse limits is not exact. According to nLab there is something called strong homology which tries to remedy this which you might want to take a look at.
EDIT: If you want something that can work for discrete spaces with additional structure, note that you can take the nerve of one specific covering, rather than taking a limit over all covers. For instance, I would guess that your homology theory of locally finite metric spaces is the same as the Cech homology of the cover consisting of the balls $dN_1(x)$ for all $x$. For "nice" spaces a similar phenomenon occurs: the limit over all covers coincides with what you get from a single cover satisfying some simple condition. For example, for simplicial complexes, you can take the cover consisting of the open star of each vertex, or for a Riemannian manifold, you can take a cover consisting of geodesically convex sets.