1) The eta invariant itself depends on the metric, but the relative eta invariant is in many cases (see comments) a homotopy invariant. The relative eta invariant is defined to be the difference of the eta invariants associated to the Dirac operator twisted by two different flat Hermitian bundles (i.e. unitary representations of the fundamental group).
2) The relation between the eta invariant and Chern-Simons invariants is a little bit subtle, but it is explained in detail in section 4 of "Spectral Asymmetry and Riemannian Geometry II" by A-P-S.
3) Arguably the most important examples are lens spaces - this is how it was first realized that the defect in the signature theorem for manifolds with boundary is non-local, for example (if it were local it would be multiplicative for coverings). There is also an interesting paper called "Eta Invariants, Signature Defects of Cusps, and Values of L-Functions" by Atiyah, Donnelly, and Singer in which the eta invariant associated to the signature operator on a Hilbert modular variety with the cusps chopped off is calculated in terms of values of Shimazu L-functions. This was apparently one of the motivating examples for the theory of eta invariants, but I don't know what actual arithmetic significance it has.