This is far from an answer, but if no answer is found I thought a partial answer might be useful.
- Since the last point (switching annotations on and off) seems to me the most difficult, Google Wave with its time slider comes to mind. Unfortunately, LaTeX support was extremely bad in my experience. But since the Apache foundation might take over the development there's at least a future.
EDIT: Dec 14, 2010 I recently remembered this questions when I came across a tool for Wordpress.
- http://digress.it/ a plugin for comments per paragraph.
- There's a number of LaTeX plugins, e.g. http://wordpress.org/extend/plugins/mathjax-latex/
- There's a plugin for comment rating http://wordpress.org/extend/plugins/comment-rating/
That seems to be the best base for a multi-user approach -- a couple of more plugins could turn it into a social reading platform.