There is no downside to arXiv. If you don't want to post a paper there, then don't. If you don't want to read papers on arXiv or to study the daily collection of abstracts of papers just posted, then don't. Nor is there any issue of plagiarism of ideas from arXiv that is different from plagiarism from a journal article. To publish an article means to make it public. A manuscript uploaded to arXiv is a publicly accessible and, therefore, a published manuscript, with the author's name attached to all the ideas, theorems, proofs, and conjectures that the paper contains.
There is also an important democratizing aspect of arXiv. In many fields of mathematics, preprints used to be circulated privately among a small group of insiders and their students. It was difficult for outsiders to learn about new results and problems, and to "break into" the club. It was essentially impossible for mathematicians in many poor countries to have access to research papers; the mathematicians were isolated and their universities could not afford journal subscriptions. Now all they need is internet access to find out about current research. The establishment of arXiv as a central repository for preprints in all fields of mathematics is an extraordinarily significant accomplishment, and will have a profound impact on the future of mathematics.
