Kähler metric on a Zariski open subset of a non-Kähler manifold I may be missing an obvious example, but here goes...
Let $X$ denote a complex manifold of dimension $n$. If $X$ is Kähler, then the induced metric on any complex submanifold is also Kähler. If instead $X$ is non-Kähler, we can at least say that any coordinate neighborhood $(U,\varphi)$ inherits a Kähler metric from the induced Euclidean metric on an open subset of $\mathbb{C}^n$. If $X$ is already equipped with a metric, note that this metric on $U$ may not be compatible with it. However, I am unsure of what can be said about the metric structure of $U$ if instead we allow something a little more exotic. For example, what if we take $U$ to be the complement of an analytic set? This is the situation in which I am most interested.
My question is:


Is there an example of a non-Kähler manifold, $X$, and a Zariski open subset $U\subset X$, such that $U$ admits a Kähler metric?


Added: Francesco has provided a class of examples in dimensions $\geq 3$ and BS has provided an example in dimension 2 as a comment to Benoît's answer. I am going to write up BS's example, with a few minor additions, as an answer. In case he decides to write his own answer, I'll delete mine and encourage everyone to vote his up.
 A: This example is due to BS -- see the edit to the original question. 
 For an example in dimension two, consider the Hopf surface $X = \left(\mathbb{C}^2\setminus \{0\}\right)/\mathbb{Z}$, where $\mathbb{Z}$ acts as $n\cdot(x,y) = (2^nx,2^ny)$. One can show that $X$ is diffeomorphic to $S^3\times S^1$. It follows that $X$ is non-Kähler because $H^2(X,\mathbb{Z}) = 0$. The $\mathbb{Z}$ action restricts to an action on $\mathbb{C}^*$, which is identified with $\mathbb{C}^*\times \{0\}$, and the smooth curve $E = \mathbb{C}^*/\mathbb{Z}$ of genus 1 includes into $X$ as a submanifold. $Y = X\setminus E$ is then biholomorphic to $E\times \mathbb{C}$ via the map $[(x,y)]\rightarrow \left([y],x/y\right)$. Thus $Y$ admits a Kähler structure. In fact, $Y$ is a quasiprojective variety, as it is isomorphic to an open subset of $E\times \mathbb{P}^1$. 

 More generally, the Hopf manifolds $X_n = \left(\mathbb{C}^n\setminus \{0\}\right)/\mathbb{Z},\ \  n\geq 2$, are all non-Kähler, as $X_n\simeq S^{2n-1}\times S^1$. The identification $\mathbb{C}^{n-1}\simeq \mathbb{C}^{n-1}\times \{0\}\subset \mathbb{C}^n$ gives rise to an embedding $X_{n-1}\subset X_n$ and, by an argument identical to the one above, we get an isomorphism $Y_n = X_n\setminus X_{n-1} \simeq E\times \mathbb{C}^{n-1}$. Again, $Y_n\subset E\times \mathbb{P}^{n-1}$ is quasiprojective. This gives an example in every dimension $n\geq 2$. 

A: One can consider the following example.
A Moishezon manifold $M$ is a compact connected complex manifold such that the field of meromorphic functions on $M$ has transcendence degree equal to the complex dimension of $M$. Complex algebraic varieties have this property, but the converse is not true if the dimension is at least $3$.
In 1967, Boris Moishezon showed that a Moishezon manifold is a projective algebraic variety if and only if it admits a Kaehler metric. Now take a non-projective (hence non-Kaehler) Moishezon manifold $M$. One can prove that such a manifold admits a bimeromorphic modification $\pi \colon \widetilde{M} \to M$ such that $\widetilde{M}$ is projective. 
This implies that there exist an analytic subset $S \subset M$ such that $U:=M - S$ is biholomorphic to an open subset of a projective variety (one can take as $S$ a $1$-codimensional analytic subset). 
Hence $U$ admits a Kaehler metric. 
For further details see the paper by Shanyu Ji Currents, metrics and Moishezon manifolds, Pacific J. Math. Volume 158, Number 2 (1993), 335-351.  
A: You might be interested in the theory of complex manifolds of class C, due to Fujiki: a complex manifold is of class C if it is bimeromorphic to a compact Kaehler manifold. The concept was introduced in:
Fujiki, A., Closedness of the Douady spaces of compact Kahler spaces, Publ.
RIMS, Kyoto Univ., 14 (1978), 1-52.
Class C manifolds have Kaehler metrics on Zariski dense open sets, by definition, but those metrics also compactify to compact manifolds, so class C is stronger than what you are looking for, apparently. Poon constructed self-dual metrics on compact 4-manifolds for which the twistor space is in Fujiki's class C, so there are interesting and important examples.
to a compact Kahler manifold), but not Kahler.
Y. S. Poon, Compact self-dual manifolds with positive scalar curvature, J. Differential Geometry 24 (1986) 97-132.
