Pontryagin wrote a book "Biography of Lev Semenovich Pontryagin, a mathematician, composed by himself". It is available online at http://www.ega-math.narod.ru/LSP/book.htm, in the original Russian. Google does a fairly good job of translation, although it refuses to translate the individual chapters completely because of their length.

The book is full of intrigue, and Pontryagin's frankness about himself and what he witnessed is astounding. I think it is fair to say that most people would not share publicly the things he is describing. 

The only relevant mention of Arnold that I could find in that book is in chapter 5. He states that in 1974 Arnold was not allowed to leave the country to lecture abroad, and that there was a conflict about this with the Executive Board of the IMU, who insisted that he should. From this, you could guess the reasons for blocking Arnold's Fields medal, if the story is true.