P = NP does not imply P^A = NP^A for every set A because for some A, NP^A allows more schemes of solutions than P^A does (via non-determinism). In particular, it is known for some A and B that P^A = NP^A and P^B not = NP^B.
The reason for confusion here may result from the fact that the notation NP^A is misleading (or outright illogical); it would be less confusing if N(P^A) were used, instead.
Then it would be more clear why P = NP does not necessarily imply P^A = N(P^A).
Or, if one would like to really dot the "i", P(TM) should be used in lieu of P, P(NTM) in lieu of NP, P(TM^A) in lieu of P^A, and P(NTM^A) in lieu of NP^A, with PM meaning Turing machines and NTM - non-deterministic Turing machines, in obvious sense.
Then it would become clear that any of the two axioms P(TM) = P(NTM) or P(TM) not = P(NTM) does not logically entail P(TM^A) = P(NTM^A) or P(TM^A) not = P(NTM^A) simply because the classes TM and NTM are not equal (despite the fact that they have the same "computational power") and, trivially, P(X) = P(Y) for some X, Y not equal to each other.
The above is true under assertion that P, NP and similar (derivative) sets are fairly arbitrary (except, perhaps, P \subseteq NP, etc.). Otherwise, if P = NP is true then P non = NP (vacuously) proves everything, and so does P = NP if P non = NP is true.