Let $f$ be any modular form of weight $k$ whose coefficients at the cusp $\infty$ lie in $\mathcal{O}_K$. Then (using GAGA and the $q$-expansion principle - see for example Katz's article on $p$-adic forms) $f$ gives rise to a section of the sheaf $\omega^k$ on $X_0(N)$ (or $X_1(N)$ + fixed under the diamond operators if you want to keep you moduli spaces nice) that is defined over the ring $\mathcal{O}_K[1/N](\mu_N)$.
Using the geometric description of $q$-expansions at all cusps in terms of the value of the geometric modular form on the Tate curve with level structure, one sees that in fact all $q$-expansions have coefficients in this ring (and are, as you say, generally expansions in $q^{1/N}$). This is simply because both $f$ and the Tate curve with these level structures are defined over this ring. This should answer your first question in the affirmative.
As for your second, I'm a bit confused because your conclusion is roughly what I would take to be the definition of "the corresponding modular form over $\overline{\mathbb{F}}_\ell$ is cuspidal."
Do you only mean to assume that the $q$-expansion at $\infty$ (making no assumptions at other cusps) has no constant term modulo $\ell$?
EDIT based on clarifications in the comments
Addressing the clarified second question, the $q$-expansions one obtains from the geometric modular form by evaluating at the Tate curve with level structure really are the $q$-expansions of the original form (all this assumes one has embedded $K$ into $\mathbb{C}$ already, but I gather that the OP has done that from the phrasing of his question). In particular, their reductions modulo $\lambda$ coincide, so if the original form has the property that all of the constant terms in the $q$-expansions are divisible by $\lambda$, then the reduced mod $\lambda$ geometric modular form is cuspidal.
