To answer your questions.
1) No, for example, $k[x^2, x^3, xy, x^2y, y^2, xy^2, y^3] \subseteq k[x,y]$ induces a bijection on points. The former is also regular in codimension 1. It is worth noting that you can make these projective without much work.
If you mean honest points and not just geometric ones, then
$$\mathbb{R}[x, y, ix, iy] \subseteq \mathbb{C}[x,y]$$
also induces a bijection on points and is an isomorphism away from the origin.
2) The condition you are probably looking for is called seminormality. If $Y$ is seminormal and you are working in characteristic zero, then what you are looking for holds.
EDIT: However, BIJECTIVE is not enough in the non-projective case (so it doesn't really matter to you), in general one should also assume that the map $X \to Y$ is finite/proper (see the Erratum to the Leahy-Vitulli paper I list below).
Here's a definition of seminormal.
Definition (Traverso, Greco-Traverso): An excellent scheme $Y$ is seminormal if every finite morphism $X \to Y$ which
- is a bijection on points, and
- induces isomorphisms at all residue fields
is an isomorphism.
When working over algebraically closed fields of characteristic zero, only the first condition matters.
Things like nodes and more generally normal crossings singulities are always seminormal.
Cusps are not seminormal (they are the canonical example of non-semi-normality).
In fact, semi-normality also behaves well with respect to Zariski's main theorem and Stein factorization constructions because if $X$ is semi-normal, then $\Gamma(X, \mathcal{O}_X)$ is seminormal as a ring (it's $\text{Spec}$ is seminormal). A very algebraic definition of a seminormal ring $R$, which makes this trivial to see, is as follows.
Definition (Swan): A reduced Noetherian ring $R$ is said to be seminormal if whenever there are elements $a, b \in R$ such that $a^2 = b^3$, then there exists an element $c \in R$ such that $c^3 = a$ and $c^2 = b$.
The main sources for learning about seminormality are probably
- Greco-Traverso, ``On seminormal schemes''.
- Swan, ``On seminormality''.
- Leahy-Vitulli, ``Seminormal rings and weakly normal varieties''.
- Vitulli, ``Weak Normality and Seminormality'' (a survey paper, but quite algebraic).