For an overview of applications of p-adic numbers in physics I would refer to the two links I added in the comment field, and to this <A HREF="https://ncatlab.org/nlab/show/p-adic+physics">nLab entry.</A> Regarding the second question "*What is the most convincing justification in physics that we need to work over the field of real or complex numbers*" I would like to quote Freeman Dyson in <A HREF="http://www.ams.org/notices/200902/rtx090200212p.pdf">Birds and Frogs:</A>

> When I look at the history of mathematics, I see a succession of
> illogical jumps, improbable coincidences, jokes of nature. One of the
> most profound jokes of nature is the square root of minus one that the
> physicist Erwin Schrödinger put into his wave equation when he
> invented wave mechanics in 1926. Starting from wave optics as a model,
> he wrote down a differential equation for a mechanical particle, but
> the equation made no sense. The equation looked like the equation of
> conduction of heat in a continuous medium. Heat conduction has no
> visible relevance to particle mechanics. Schrödinger’s idea seemed to
> be going nowhere. But then came the surprise. Schrödinger put the
> square root of minus one into the equation, and suddenly it made
> sense. Suddenly it became a wave equation instead of a heat conduction
> equation. And Schrödinger found to his delight that the equation has
> solutions corresponding to the quantized orbits in the Bohr model of
> the atom.
> 
> It turns out that the Schrödinger equation describes correctly
> everything we know about the behavior of atoms. It is the basis of all
> of chemistry and most of physics. **And that square root of minus one
> means that nature works with complex numbers and not with real
> numbers.**
> 
> All through the nineteenth century, mathematicians from Abel to
> Riemann and Weierstrass had been creating a magnificent theory of
> functions of complex variables. They had discovered that the theory of
> functions became far deeper and more powerful when it was extended
> from real to complex numbers. But they always thought of complex
> numbers as an artificial construction, invented by human
> mathematicians as a useful and elegant abstraction from real life. It
> never entered their heads that this artificial number system that they
> had invented was in fact the ground on which atoms move. They never
> imagined that nature had got there first.