Pontryagin wrote a book "Biography of Lev Semenovich Pontryagin, a mathematician, composed by himself". It is available online at http://www.ega-math.narod.ru/LSP/book.htm, in the original Russian. Google does a fairly good job of translation, although it refuses to translate the individual chapters completely because of their length.
TheIn the book is full of intrigue, and Pontryagin's franknessPontryagin shares a lot about himselfthe inner workings of the IMU Executive Board and what he witnessed is astoundinghis own role in holding the Soviet party line there as its vice president. I think it is fair to sayFor example, he recounts his version of how France got the IMU presidency in 1974, so that most people would not share publiclyneither the things he is describingSoviet Union nor the US would dominate.
The only relevant mention of Arnold that I could find in that book is in chapter 5. He states that in 1974 Arnold was not allowed to leave the country to lecture abroad, and that there was a conflict about this with the Executive Board of the IMU, who insisted that he should. From this, you could guessextrapolate the reasons for blocking Arnold's Fields medal, if the story is true.