This example is due to BS -- see the edit to the original question.
For an example in dimension two, consider the Hopf surface $X = \left(\mathbb{C}^2\setminus \{0\}\right)/\mathbb{Z}$, where $\mathbb{Z}$ acts as $n\cdot(x,y) = (2^nx,2^ny)$. One can show that $X$ is diffeomorphic to $S^3\times S^1$. It follows that $X$ is non-Kähler because $H^2(X,\mathbb{Z}) = 0$. The $\mathbb{Z}$ action restricts to an action on $\mathbb{C}^*$, which is identified with $\mathbb{C}^*\times \{0\}$, and the smooth curve $E = \mathbb{C}^*/\mathbb{Z}$ of genus 1 includes into $X$ as a submanifold. $Y = X\setminus E$ is then biholomorphic to $E\times \mathbb{C}$ via the map $[(x,y)]\rightarrow \left([y],x/y\right)$. Thus $Y$ admits a Kähler structure. In fact, $Y$ is a quasiprojective variety, as it is isomorphic to an open subset of $E\times \mathbb{P}^1$.
More generally, the Hopf manifolds $X_n = \left(\mathbb{C}^n\setminus \{0\}\right)/\mathbb{Z},\ \ n\geq 2$, are all non-Kähler, as $X_n\simeq S^{2n-1}\times S^1$. The identification $\mathbb{C}^{n-1}\simeq \mathbb{C}^{n-1}\times \{0\}\subset \mathbb{C}^n$ gives rise to an embedding $X_{n-1}\subset X_n$ and, by an argument identical to the one above, we get an isomorphism $Y_n = X_n\setminus X_{n-1} \simeq E\times \mathbb{C}^{n-1}$. Again, $Y_n\subset E\times \mathbb{P}^{n-1}$ is quasiprojective. This gives an example in every dimension $n\geq 2$.