This is far from an answer, but if no answer is found I thought a partial answer might be useful.
- Since the last point (switching annotations on and off) seems to me the most difficult, Google Wave with its time slider comes to mind. Unfortunately, LaTeX support was extremely bad in my experience. But since the Apache foundation might take over the development there's at least a future.
- On the other hand Tiddlywiki has most of the features you're looking for. Above all, it is very hackable with tons of plugins (excellent LaTeX plugins like MathSVGPlugin and jsmath). Of course, its technology (javascript, single file) does not scale well but it might offer some ideas. I use it in several projects with a small number of users. There's a wonderful copy of the Tractatus as an example for its abilities.
EDIT: Dec 14, 2010 I recently remembered this questions when I came across a tool for Wordpress.
- http://digress.it/ a plugin for comments per paragraph.
- There's a number of LaTeX plugins, e.g. http://wordpress.org/extend/plugins/mathjax-latex/
- There's a plugin for comment rating http://wordpress.org/extend/plugins/comment-rating/
That seems to be the best base for a multi-user approach -- a couple of more plugins could turn it into a social reading platform.
