I am interested in clean algorithms for approximating solutions and so I am interested in numerical analysis, but most of the books I have seen get bogged down in error analysis or they spend a lot of time and effort in squeezing an additional 2% efficiency from a classical algorithm. What are some works which just have the fundamental ideas and methods?

1$\begingroup$ What sort of solutions are you approximating? The amount of error control you have to do to get a meaningful answer depends strongly on the problem domain. $\endgroup$ – S. Carnahan♦ May 11 '10 at 13:40

4$\begingroup$ I always feel a bit awkward recommending a book with which I am heavily associated, but Nick Trefethen wrote a beautiful article on the general ideas of numerical analysis for the Princeton Companion to Mathematics. It's even available online: comlab.ox.ac.uk/nick.trefethen/NAessay.pdf $\endgroup$ – gowers Jun 8 '10 at 15:37

$\begingroup$ It has been 4 weeks now. Voting to close... $\endgroup$ – S. Carnahan♦ Jun 8 '10 at 15:50

$\begingroup$ This is a very broad question. Are you seeking approximations of solutions of PDE? Large linear systems? Eigenvalue problems? Optimization problems? Integral equations? Any book which professes to do all of these will not do any in depth. A collection of 'just the methods' would, for example, be 'Numerical Recipes in C', but is this the level that you want? $\endgroup$ – Nilima Nigam Jun 1 '11 at 4:59
Are you looking for a reference that links the field of numerical analysis to mathematical concepts moreso than algorithmic concepts? Matrix Computations by Golub and Van Loan is a fairly important book that studies the algebraic structures of matrices and derives algorithms from those properties. If you're looking for an entrylevel work, I keep a copy of Michael Heath's book Scientific Computing on my desk. It covers fundamental concepts and algorithms fairly well, in my opinion.
Do you have a specific problem domain in mind?
To augment Timur's answer:
Claes Johnson's introductory book on FEM
Braess's book on FEM
Iserles's book on numerical analysis of DE
Gottlieb and Orzsag's book on spectral methods
Nick Trefethen's book on spectral methods for spectral collocation ideas.
Quarteroni, Sacco, Saleri on numerical methods.
From 'the horse's mouth', the Cleve Moler book on numerical computing using Matlab.
I've picked these books for their balance of important algorithms and key insights, delivered with clear prose.
I also like Strikwerda's book on finite difference methods, and the HairerWanner books on numerical methods for ODE. But these focus a lot on error analysis, which may not be what you wish.
One of my favorites is "Numerical Methods that Work" by Forman S. Acton. It's an old book; some of the examples seem quaint now that we have far more compute power. But the principles haven't changed.
Acton's book is good about illustrating technique, learning how to think like a numerical analyst.

$\begingroup$ The sequel "Real Computing Made Real" was written in the same spirit; just think of Acton here as a Dutch uncle: "You did what? What the hell?!" :) $\endgroup$ – J. M. is not a mathematician Sep 20 '10 at 22:03
My favorites:
 Lloyd Nicholas Trefethen, David Bau. Numerical linear algebra
 Randall J. LeVeque. Finite difference methods for ordinary and partial differential equations
 Randall J. LeVeque. Numerical methods for conservation laws
 Susanne C. Brenner and L. Ridgway Scott. Mathematical theory of finite element methods