The answer is no. Let $\kappa$ be any infinite cardinal, regular or singular, and assume for a contradiction that there is a set $E\subseteq\mathcal P(\kappa)$ satisfying your conditions. I will call the elements of $\kappa$ points and the elements of $E$ lines.
First note that there are $\lt\kappa$ lines through any point. For consider any point $\alpha$ and let $\lambda$ be the number of lines through $\alpha$.Choose a point $\beta\ne\alpha$ and a line $e$ which is incident with $\beta$ but not with $\alpha$. Since each line incident with $\alpha$ meets $e$ in a different point, $\lambda\le|e|\lt\kappa$.
Now choose two distinct points $\alpha$ and $\beta$. Say there are $\lambda$ lines through $\alpha$ and $\mu$ lines through $\beta$. Let $e$ be the line through $\alpha$ and $\beta$. Now every point which is not on the line $e$ is the point of intersection of a line through $\alpha$ and a line through $\beta$. Hence $\kappa\le|e|+\lambda\mu\lt\kappa$ which is absurd.
This argument is adapted from the proof that a finite projective plane of order $n$ has $n^2+n+1$ points. In that case we have $|e|=\kappa=\lambda=n+1$ and the number of points is $|e|+(\lambda-1)(\mu-1)=n^2+n+1$.