The group $U(\mathbb Z)$ is nilpotent, but not isomorphic to $\mathbb Z^3$.
The quotient space is not the torus $T^3$ (= cube with opposite sides identified).
The fundamental domain which glues together to make the quotient can be described, but I'm not sure it's what you really want or need: there are easier ways to understand it.

The element $c $ with the 1,3 entry equal to 1 generates the center of $U(\mathbb Z)$, with quotient $\mathbb Z^2$, and it is the commutator of the other two elementary matrices $a$ and $b$. Geometrically, the
quotient has a map to a torus $T^2$ (factoring out by your troublesome entry) with fiber (preimage of any point) a circle, but the circles form a fiber bundle that is not trivial.

You can map all this into R^3 as a group of affine transformations, and visualize
how it works: $c$ acts as a vertical translation.
The element $a$ translates in the $x$ direction by one unit, but at the same time shears planes perpendicular to the x axis so the y-axis develops a slope of -1/2. The element $b$
is similar, translating in the $y$ direction, taking vertical lines to vertical lines and shearing the $x$ axis to have slope 1/2. Now look at what happens when you do the commutator
$a^{-1} b^{-1} a b$. This group is also called the Heisenberg group.

In higher dimensions, the picture is similar but more elaborate, with iterated circle bundles over previous stuff, exhibiting additional shearing. (There is more than one reasonable way to choose coordinates, so don't be surprised if you don't get the exact same description when you work through details).

You can find pictures and descriptions in my book on Three-dimensional Geometry and Topology (among other places). It's the basic example for one of the 8 geometries for 3-manifolds. I first learned this picture by reading Poincare.

**Addendum**
In response to the added questions:
Keivan Karai and Max have addressed most of it, in comments to the question.
It's true, there are many sources to read about these things. Many people have studied the geometry of $SO(3)\SL(3,\mathbb R)/SL(3,\mathbb Z)$, it's challenging to come to grips with it completely, but a vast amount is known.

You can interpret $SO(3)\SL(3, \mathbb R)$ fairly concretely as the space of positive definite quadratic forms of determinant 1 on $\mathbb R^3$, and it has a natural metric in which a path of quadratic forms has arc length equal to the total strain it takes to do the associated deformations --- you can imagine an object, perhaps a ball, made of a highly viscous and inelastic material, perhaps modeling clay, that you can squeeze into different shapes that don't spring back. You're squeezing the ball into different ellipsoid shapes via linear transformations, and keeping track of how much energy it takes. (use the $L^2$ norm of the first derivative, as measured in using the current shape).

In 3 dimensions, quadratic forms have 6 coefficients, but the condition that the determinant is 1 brings it down to 5. To think about $SO(3)\SL(3,\mathbb R)/SL(3,\mathbb Z)$, imagine a 3-torus with a flat metric, that is, $\mathbb R / \mathbb Z^3$ for some action of $\mathbb Z$.
The set of possible shapes for the torus, up to isometry is how you can interpret it.
(This is beccause $\SL(3,\Z)$ is the group of affine automorphisms of the torus, up to translations, so it acts on the set of Euclidean metrics on a torus with fixed basis --- in the double coset space, we're forgetting the basis).

The Heisenberg group gives the small scale geometry of the quotient when you go off to infinity in certain directions --- that is, if you have a torus with one loop $x$ of size $l_1 < \epsilon$ and another loop $y$ size $l_2 < l_1 \epsilon$, then in the moduli space for toruses, the subgroup of $SL(3,\mathbb Z)$ generated by low-energy loops is isomorphic to the Heisenberg group. By making $\epsilon \rightarrow 0$ the geometric size of this cross-section 3-manifold ( where the shortest and second shortest lengths are held constant)
get arbitrarily small. This is *part* of the picture of the space near infinity, but to get a fuller picture, you need to examine the global structure of how different behaviors at infinity connect to one another.

There's a lot more needed before you have a very good picture of what's known about
$SL(3,Z)$ or of this locally symmetric space --- a lot is known from a number of different points of view, and it's challenging to fully understand. This isn't the place for an exposition, even if I knew all the answers (which I don't).