There are many approaches to this problem.
The subjectivist approach says the prior should simply quantify what is known/believed before the experiment takes place. Period. End of discussion.
The empirical Bayes approach says you should estimate your prior from the data itself. (Obviously in that case your "prior" isn't prior at all.)
The "objective" Bayes approach says to pick priors based on mathematical properties, such as "reference" priors that in some sense maximize information gain. Jim Berger gives a good defense of objective Bayes here.
In practice someone may use any and all of these approaches, even within the same model. For example, they may use a subjective prior on parameters where there is a considerable amount of prior knowledge and use a reference prior on other parameters that are less important or less understood.
Often it simply doesn't matter much what prior is chosen. For example, you might show that a variety of priors, say an optimistic prior and a pessimistic prior, lead to essentially the same conclusion. This is particularly the case when there's a lot of data: the impact of the prior fades as data accrue. But for other applications, such as hypothesis testing, priors matter more.