Let $H$ be a CM field and $F$ be the maximal totally real subfield of $H$. Can we construct a Katz $p$-adic L-functions of Hecke characters without the ordinary condition (i.e every prime of $F$ above $p$ splits in $M$)?

A p-adic L-function is expected to depend on (at least) two pieces of data: a family $V$ of representations of $G_{\mathbf{Q}}$ over some base space $X$ (which should be a p-adic formal scheme or rigid space); and a family of subspaces $V^+$ of $V$ stable under $G_{\mathbf{Q}_p}$ (a "p-refinement" or "p-stabilisation"). (It's actually sufficient to have a submodule of the Robba-ring $(\varphi, \Gamma)$-module of $V$ at $p$, rather than of $V$ itself, and there's also a choice of periods which I'm sweeping under the carpet, but let me ignore those for now).

The pair $(V, V^+)$ are required to satisfy the "Panchishkin conditions":

- $rk(V^+) = rk(V^{c = -1})$ where $c$ is complex conjugation;
- there is a Zariski-dense set of points $\Sigma \subset X(\mathbf{C}_p)$ such that $V$ is de Rham at $p$, $(V^+)_x$ has all its Hodge--Tate weights $\ge 1$, and $(V/V^+)_x$ has all its Hodge--Tate weights $\le 0$.

Then one expects to attach to $(V, V^+)$ a p-adic $L$-function over $X$, whose value at a "Panchishkin point" $x \in \Sigma$ is the L-value at 0 of a motive whose $p$-adic realisation is $V_x$ -- this is a critical value, because of the condition on the Hodge--Tate weights -- modified by appropriate local factors depending on the $L$ and $\varepsilon$ factors of $V^+$ at $p$.

Now, in your case you're taking $X$ to be the character space of the ray class group of $H$ modulo $p^\infty$, and $V$ to be the induced rep from $H$ up to $\mathbf{Q}$ of the natural 1-dimensional family over $X$. So what are the subrepresentations $V^+$ that one can take? The subreps of $V$ correspond precisely to subsets of the primes of $V$ above $p$, and you have to choose a set where the sum of the local degrees is $\tfrac{1}{2}[H : \mathbf{Q}]$.

But what will the points $\Sigma$ be? This is the tricky part! If $\psi$ is a Groessencharacter of $H$, then $\psi \psi^*$ will factor through $F$, and since $F$ is totally real, this forces $\psi \psi^*$ to have **parallel** infinity-type (i.e. $\psi\psi^*$ is a finite-order character times a power of the norm). It's not too hard to see from this that if the set of primes you chose is not disjoint from its image under complex conjugation, then the set of points of $X$ where the Panchishkin condition is satisfied is actually empty!

So if you believe a certain set of prescriptions about what $p$-adic $L$-functions ought to look like, then you can rule out the existence of a version of the Katz L-function when Katz's ordinarity condition is not satisfied. Of course, there could be more general notions of "$p$-adic $L$-function" which are not ruled out by this; but I hope it at least goes some way towards explaining what this assumption is there for.