For math books you probably need to look at larger devices (8" or 9"):

 - iRex DR1000s and DR800SG (am not sure the company still exists) 
 - The new Kindle DX Graphite
 - The iPad (not a reader, but it's here)

However, I find that that 6" devices are more portable, unlike the (more bulkier?) 8" or larger screens. But having a smaller screen means less screen estate and the PDF format cannot reflow correctly because the page layout is designed for a specified size and is constrained by graphics and positioning of other elements. This means that what you see on the screen is a smaller version of the page, sometimes not easy on the eyes or downright illisible. The solution is to read in landscape node and if possible to cut the page margins in the PDF.

Just for PDF and dejaview, take a look at Pocketbook 301+. It is, in my opinion, the best PDF reader in the 6" class. There is newer model (Pocketbook 302) but it has a glossy touch screen, so I would avoid that. I use it mostly for technical documents, heavy mathematics with graphics and it's great because:

1. Zoom to arbitrary level and keep the zoom between page switches (persistent zoom)
2. Landscape mode with automatic cut margins
3. Multiple-column mode (great for those two colums papers A4 or Letter sized papers)
4. Excellent DejaView viewer

The PB301+ is a EB600EM clone with 8 grayscale levels (revised in 2009). Do not mistake it with other EB clones (there is a gazillion of them): the firmware is what makes Pocketbook a great PDF viewer and you cannot install it on other clones.

Concerning larger device: there is a number of them arriving soon on the market, including the Pocketbook 901 (9.7" screen size) expected in september 2010.

Beware of the touch screens: they reflect a lot (screen glare). I had a Sony PRS600 but I sent it back. I really wanted this to be a non-issue, but the glare and the reflections ended by getting in my way when reading: I needed a constant conscious effort to not see the reflection of my face in the device, all this combined with a sort of "fuzzy display" due probably to the "through the glass" effect induced by the screen.

The motherlode site of mobile readers is http://wiki.mobileread.com (the technical descriptions are here: 
http://wiki.mobileread.com/wiki/E-book_devices). The forums are a good source of personal opinions.

Now, surprisingly, reading mathematics on a 6" screen turn out to be a much better experience that I have thoought. This is subjective, but again, I expected worse. Here are some of my impressions (for a PB301+ 6" screen):

 - Landscape mode is OK: you will see one third or half a page at a time but you can switch quite fast up and down in the same page
 - Page turn is slower (1 sec), so turning pages back and forth is not practical. You can jump directly to any page but you will have to type the page number, again not practical...
 - Bookmarks are great and allows you to jump anywhere in the document.
 - Reading is just like paper, not very stressful for the eyes.

Good luck in your choice!