Is there a name and common technique for such equations, where $A$ and $B$ are matrices and $x$ a vector?
Akin to my comment, this equation can be called a nonlinear generalized eigenvalue problem. Usually, $f$ and $g$ are polynomials in $\lambda$, but more general nonlinearities might be allowed. In general, I doubt there will be robust, globally convergent method for this equation that gets all the solutions. The talk or this paper might be good starting points (see especially the paper).