There are indeed counterexamples to which Igor Belegradek gave reference. Here is another counterexample in the plane, perhaps the simplest there is: Let $X$ be an annulus with one arc attached to one of its boundary components and another arc attached to the other boundary component, and $Y$ - an annulus with two disjoint arcs attached to the same one of its boundary components.
