There are many approaches to this problem. Here are three.

 1. The **subjective** approach says the prior should simply quantify what is known or believed before the experiment takes place. Period. End of discussion.

 2. The **empirical** Bayes approach says you should estimate your prior from the data itself. (Obviously in that case your "prior" isn't *prior* at all.)  

 3. The **objective** Bayes approach says to pick priors based on mathematical properties, such as "reference" priors that in some sense maximize information gain.  Jim Berger gives a good defense of objective Bayes <a href="http://www.stat.duke.edu/~berger/papers/obayes-debate.pdf">here</a>. 

In practice someone may use any and all of these approaches, even within the same model. For example, they may use a subjective prior on parameters where there is a considerable amount of prior knowledge and use a reference prior on other parameters that are less important or less understood.

Often it simply doesn't matter much what prior you use. For example, you might show that a variety of priors, say an optimistic prior and a pessimistic prior, lead to essentially the same conclusion. This is particularly the case when there's a lot of data: the impact of the prior fades as data accrue. But for other applications, such as hypothesis testing, priors matter more.