How to make the move to a research university from a liberal arts school? Some mathematicians are in positions that emphasize teaching at predominantly undergraduate colleges but find themselves desiring positions at research universities.  This may happen because their career preferences have changed or because they always wished for research oriented positions but were unable to find one earlier.  Having said this, I'll ask:

1) Precisely how does one move from a teaching oriented job to a research oriented job? I'm aware that one should build relationships at universities and should write good papers, but, given the teaching and service expectations at most predominantly undergraduate institutions, is it really reasonable to expect that enough solid work can be done to make one's CV competitive?

The prevailing wisdom is that one should try to make such a move before getting tenure at a predominantly undergraduate institution. So, additionally, I'll ask:

2) Is it possible to move from a teaching job to a research job after receiving tenure at a predominantly undergraduate institution? If so, what advice do you have for someone trying to do this?

 A: I can't answer this question, but perhaps can offer some info:
Edit 1: I realized that what appears below would perhaps be more suitable as an answer to a question like: "How do you cope with the reality of being at an undergraduate-focused school when you had your heart set on a research career?" This is markedly different from the OP's question, but evidently my answer seems useful for this one. Perhaps I've implicitly answered the question...It would be nice if this question received an actual explicit answer! (e.g. I think Joseph O'Rourke's deleted answer was closer to the mark than mine is!)
1') Important Edit 2 Recent economic pressures on undergraduate colleges connected with projected enrollment decline, health insurance laws, public perception of higher education and resulting changes in "product delivery", has led me to revisit the advice I gave in (1) below. It is very unlikely that any but perhaps a few elite undergraduate liberal arts colleges will be able to afford to support research substantially in a sustainable way. Small class sizes and reduced teaching loads are regarded by most colleges as mutually exclusive and unaffordable to maintain simultaneously. Since the former is a big selling point of the liberal arts college, the latter probably will not easily be found at a liberal arts college without financial problems at the institution following closely behind, if the institution is one of the many that are primarily enrollment-driven. If you happen to enjoy research and don't want to be forced out of it by economic reasons like teaching load or salary that needs supplement, do not look for a job at a liberal arts college with less than a billion dollar endowment. (I believe this edit is important, since the old answer might mislead future job candidates for such institutions…which I really do not want to do.)
1) Not all liberal arts schools are the same. Not considering the elite schools, some departments have 3 hour loads distributed over two courses. I work at such a place. Office hours and service are still more of a commitment than at a research school, but there is still time left over to work. My college is clearly trying to raise its research profile a little. My feeling is that due to the sheer number of qualified PhDs looking for jobs this is an emerging market, and so colleges that want to move up in the rankings will consider reducing teaching loads in order to take advantage of the oversupply of strong research candidates. You won't be able to write two good papers per year...perhaps not even one good paper per year...but you will be able to do research at such a place. Chances are, if you have tenure at a small school then you have a knack for engaging undergraduates (whether you like it or not). I'll bet you will be an attractive candidate for a move to such a department.
2) This is presumptuous to write, because it's pretty obvious, but getting a good research job usually requires making very serious sacrifices. Many of my research friends still are working hard on work/life balance issues. Some have had to be separated from a spouse (often on a different continent) in order to try to get a decent research job. I can't help but think that people who land these jobs often quite deserve them. I can say that my own unwillingness to expose myself to such risk is the main reason why I originally accepted a job at a liberal arts college instead of trying to go to a research university.
3) Having just chaired a search committee, I've recently reviewed more than 750 applications for a tenure-track liberal arts job. The gut-wrenching truth is that there were more excellent research candidates than the market can support. If we restrict our attention only to those with at least one postdoc, it is probably possible to fill all openings on MathJobs with candidates having more than 5 publications (and this is conservative). Budgets allow for bringing a few candidates to campus for an interview, and almost nothing is worse for a potential employer than blowing a campus interview on someone who is not serious about a position. To some extent, your vulnerability as an applicant is attractive to the interviewer. If you have tenure already, it is possible that you will be harder to secure...or worse, will be seen as collecting offers to bargain for a raise at your current job. Unfortunately, I have strong evidence that this happens. In one case, this evidence stopped us from talking to an untenured candidate. I think that this is the main reason why it is nearly impossible to move after tenure. (Sorry to kill hope...I hope I am wrong about this!)
4) One final important point I forgot to include before. As you mourn the fact that you never will teach graduate courses or supervise PhD students, keep in mind that there is an important positive side to this fact: You will not have to help such PhD students find an academic job in the current market. I imagine that having taken a job at an undergraduate institution, if you were to relocate to a more research-based position it probably will not be Princeton. Instead, you will probably find yourself at a lower-tier research school. I speculate that PhD students graduating from such schools are at a disadvantage for landing excellent postdocs (If I'm wrong about this, someone please contradict me) and subsequently finding permanent academic employment. I imagine watching students go through this would be very stressful for a conscientious thesis advisor.  
A: I hope someone will supply a better answer, but here are some thoughts, based on one case in which I wrote a letter for someone who successfully moved from a teaching-oriented to a research-oriented job.  It was before tenure, and I really don't know a strategy for doing it post-tenure.
My impression is that there are two tricky issues.  One is that it's difficult to develop one's research abilities to their full potential in an environment not optimized for that.  It's therefore important to have letters that say you have the potential to do even better work in the right environment.  On the other hand, they also need to emphasize the quality of your current work (if not its quantity), since if they just focus on your potential you will come across as a big risk.  This takes some balancing, and it's not likely to happen unless the recommender thinks about it.  That should always happen, and it usually does, but some people just don't know how to write a good letter of recommendation, and others may be too busy to do a good job.  Unfortunately, choosing the right letter writers can be difficult, since you don't have the information you need to judge this.  I'd recommend talking over the possibilities with a trusted mentor, trying to choose recommenders who are really supportive, and then explicitly strategizing with them about your circumstances.  However, that's easier said than done.
The other issue is that you may be quickly eliminated based on superficial criteria, just because there are so many applicants that they can't each get more than a quick reading.  What you need is a mentor who can e-mail people at various universities and let them know that even though your application may not fit their preconceived notions of what they are looking for, they should examine it carefully.
