First, a historical remark: it was not until relatively recently in the history of science that people were convinced that the atomic theory of matter is correct. I believe the tide was turned by a paper by Einstein in 1905 which explained Brownian motion (as actually observed by Robert Brown) using the assumption that water is made up of molecules. Before that many scientists held the belief that the universe really is continuous, and even those who didn't had trouble arguing with the predictive and explanatory success of continuous models.

Aside from that, the premise underlying this question ignores many deep and fundamental issues associated with passing back and forth between the continuous and the discrete. The sentence "just choose a very small $\Delta x$ instead of $dx$" sweeps under the rug some profoundly difficult mathematical problems. Some examples:

• Global dynamical properties of a system are often hard to see in discrete models. For instance numerical stability issues make it very hard to discretely analyze hyperbolic systems. There are also some behaviors that just don't show up in a naive discretization - for instance, it is not at all obvious why the second law of thermodynamics is consistent with the atomic theory of gases (wherein the equations are symmetric in time).
• While there are a number of standard ways to replace an ordinary differential equation with a difference equation, the corresponding technique for partial differential equations (the finite element method) is extremely challenging and is the basis for a lot of current research in numerical analysis.
• Approximate solutions are actually not simpler than exact solutions in many (most?) cases. Consider the isoperimetric problem: find the planar curve of a given length which encloses the largest area. This can be reduced to solving a system of ordinary differential equations (the Euler equations). If you do it analytically you get a circle; if you do it discretely you get a sequence of curves which give better and better approximations of a circle. How is the latter simpler? This is a serious issue in physics: continuous models often have lots of symmetry that you lose when you discretize them.

I'll also point out that one of the hardest problems in modern mathematical physics - finding a quantum theory of gravity - has so far resisted the "just choose a very small $\Delta x$ instead of $dx$" approach.