These days the best tool for visualizing math is probably a computer.  Nevertheless, I find it helpful/fun/nice-break-from-the-computer to sometimes make physical objects or drawings by hand.  The most basic tools are of course a pen and paper, and a lot can be done with just this, especially if you start folding.  Here are some other basic supplies I keep in my office:
- Colored pencils/pens
- Cardstock
- String
- Thumbtacks
- Compass
- Straightedge
- Protractor
- Tape
- Scissors

What are some other supplies that are useful to have around?