Smooth a matrix

I have a matrix in which each element contains the coordinates of a 3D surface. Sometimes, some points will be "out of line" meaning that they will not conform to the general shape. For example you would have a slightly curved plane and all of a sudden a point with coordinates which are very different from the rest which produces a spike in the surface.

I was thinking of applying a smoothing filter or a lowpass filter.

Then I thought I might produce a distribution of the gradients of all points (w.r.t. the points around them) and only allow points with a gradient within two std. of the mean.

I'm really not sure what the best way would be (there might be others which I don't know about) so before I started implementing it into matlab I thought I'd ask.

-
I think you need to edit your first sentence to be more precise. Do you mean that each matrix element is a 3D vector giving a point on your surface? – Mark Grant Feb 22 '12 at 14:47
Yes. As a matter of fact I have 3 separate matrices but I thought it's easier to grasp if we imagine a cell where each element is a 3D position vector. – s5s Feb 22 '12 at 15:12
The best way to go will certainly depend on many factors (e.g. the type of noise you expect to have, the notion of error that you want to minimize, etc.). Also, whatever you do could be thought of as "applying a smoothing filter", so it is not clear what you mean by "there might by others [ways]". – Ramiro de la Vega Feb 22 '12 at 19:03
Another keyword you might want to check is "outliers". This is some type of noise in this setting. – Dror Atariah Feb 24 '12 at 11:02