Arnold himself confirmed what you wrote in your question:
V.I.Arnold "Yesterday and long ago", Springer, 2007 (translated from Russian), p. 94:
"...while having dinner he (L.S.Pontryagin) explained to me that he had to go to an international meeting and talk against me..."
So basically what you cited seems to be true, though Arnold did not mention that "...Pontryagin was ordered not to allow the award of Fields Medal to Arnold" (in Russian "had to" not necessary means "was ordered to").