(Too long for a comment on Gerry Myerson's answer.)
It is clear that edge-transitive implies the OP's property. But at least for infinite (multi-)graphs, I believe that edge-transitivity is strictly stronger. Consider the following infinite graph, with countably many vertices and countably many edges. There are two distinguished vertices, say $A$ and $B$, which are each connected by $\aleph_0$ edges. Moreover, there exist further vertices $\{A_n\}_{n=1}^{\infty}$, $\{B_n\}_{n=1}^{\infty}$, such that each $A_n$ is connected by $\aleph_0$ edges to $A$ and each $B_n$ is connected by $\aleph_0$ edges to $B$. In particular, whenever there is one edge between two vertices there are infinitely many, so removing any edge leaves a graph which is isomorphic to the one we started with -- a fortiori any two choices of edge removal lead to isomorphic graphs. But the edges joining $A$ to $B$ lie in the middle of a path of length three, whereas the other edges do not, so the graph is not edge-transitive.
Whether there are counterexamples which are not so "cheap" remains to be seen...