One way is to allow recursion on ordinals for ptykes for all finite types. Then the supremum of all ordinals describable in this way will be exactly B-H ordinal. See outline of ptyx interpretation of modified Gödel's system $T$ in the draft of Girard's book about ptykes [Section 12.A;1]. I believe that the details of computations that this leadleads to B-H ordinal had been carried out by Päppinghaus, but I never looked into this details myself [2]. In a sense this approach is just a generalization of "predicative" ordinal notation systems to finite types.
In our recent paper [3] Juan Aguilera, Andreas Weiermann and I defined a functor $B\colon \mathsf{Dil}\to\mathsf{Dil}$ that is very similar to $\Lambda$, but I think that our definition is considerably more compact. For $B$ we have two equivalent definitions. One definition basically is that $B(D)$ is a natural extension of a function $B_{D(\omega)}\colon \omega\to \omega$ from a version of binary fast-growing hierarchy up to $D(\omega)$ to the type $\mathsf{Ord}\to\mathsf{Ord}$. Which demonstrates the functor $B$ to be a variant of recursion along dilators. The other definition of $B$ is in the terms of a term system for certain version of an ordinal collapsing function $\psi$. Thus making aan explicit connection between Girard's idea of tame type 2 bar-recursion and ordinal collapsing. B-H ordinal is $B(\varepsilon^+,0)$, where $\varepsilon^+$ is a naturally defined dilator mapping an ordinal $\alpha$ to the smallest $\varepsilon$-number strictly above $\alpha$.
With regards to what are the limits of the approaches. The limit of the approaches based on recursion on ordinals is B-H ordinal (at least as long as we limit ourselves to finite types). Recursion on dilators seems to be closely connected to collapsing functions and although it have been studied much less than the latter, probably it will have similar limitations (the notation systems for systems that are not too much stronger than $\Pi^1_1\textsf{-CA}_0$ are fairly simple, but the extensions of the approach to $\Pi^1_2\textsf{-CA}_0$ and even weaker systems become quite complicated). I don't know about any works that extend ptyx-based approach beyond recursion on dilators. I have been working on this, but haven't yet published anything about it. There are some indications that it might be possible to reach the ordinal of full second-order arithmetic.
[4] A. Freund. $\Pi^1_2$$\Pi^1_1$-comprehension as a well-ordering principle. Adv. Math., 355, 2019