Pontryagin wrote a book "Biography of Lev Semenovich Pontryagin, a mathematician, composed by himself". It is available online at http://www.ega-math.narod.ru/LSP/book.htm, in the original Russian. Google does a fairly good job of translation, although it refuses to translate the individual chapters completely because of their length.
In the book, Pontryagin shares a lot about the inner workings of the IMU Executive Board and his own role in holding the Soviet party line there as its vice president. For example, he recounts his version of how France got the IMU presidency in 1974, so that neither the Soviet Union nor the US would dominate.
The only relevant mention of Arnold that I could find in that book is in chapter 5. He states that in 1974 Arnold was not allowed to leave the country to lecture abroad, and that there was a conflict about this with the Executive Board of the IMU, who insisted that he should. From this, you could extrapolate the reasons for blocking Arnold's Fields medal, if the story is true.