A fibered knot is a knot with a homeomorhism on compact surface with one boundary component. On the contrary, for a given homeomorohism on a compact surface with one boundary component, is there any way to determine whether it is a monodromy of a fibered knot?

Form the mapping torus $M$, check whether $H_1(M;\mathbb{Z})=1$ and is generated by a loop on the torus boundary. If not, it isn't a fibered knot complement. If so, do Dehn filling to produce a closed 3manifold. Use sphere recognition algorithms to check if the result of Dehn filling is homeomorphic to the 3sphere.  Edit  As emphasized by the comments of Agol and Igor Rivin, this is not an algorithm until one specifies exactly how to compute which Dehn filling curves in $\partial M$ need to be checked. I'll say how to do this when the monodromy $\phi : S \to S$ is pseudoAnosov, which corresponds to checking whether $M$ is a fibered hyperbolic knot complement (something similar will work to check whether $M$ is a satellite knot whose torus decomposition has a hyperbolic component incident to $\partial M$; something different needs to be done when $M$ is Seifert fibered or is a satellite knot complement whose torus decomposition has a Seifert fibered component incident to $\partial M$). First compute the longitude and the degeneracy curves $\ell,\delta \in H_1(\partial M;\mathbb{Z})$. The longitude $\ell$ is the generator of the kernel of $H_1(\partial M;\mathbb{Z}) \to H_1(M;\mathbb{Z}) \approx \mathbb{Z}$. To compute $\delta$, first compute an invariant train track $\tau \subset S$ such that some component $C$ of $S \setminus \tau$ is a crown surface containing $\partial S$; one can use standard train track algorithms to do this, such as the BestvinaHandel algorithm. Then suspend $\tau$ to get a branched surface $B$ in $M$. On $B$, take the suspension curve of a cusp of $C$. Isotope that curve through an annulus$\times [0,1]$ component of $M \setminus B$ to get $\delta \subset \partial M$. From a more invariant but less algorithmic point of view, if one suspends the stable geodesic measured lamination $\lambda \subset S$ of $\phi$ to get an essential lamination $\Lambda$ in $M$, the curve $\delta$ is the unique one which is isotopic to a curve in a leaf of $\Lambda$. Now compute the three curves on $\partial M$ whose geometric intersection with $\ell$ equals $1$ and whose geometric intersection with $\delta$ equals $0$ or $1$. Those are the only three possible curves along which Dehn filling can give a 3sphere: every Dehn filling along a curve which intersects $\ell$ either $0$ times or $\ge 2$ times has nontrivial 1st homology with $\mathbb{Z}$ coefficients; and every Dehn filling along a curve which intersects $\delta$ more than $1$ time has an essential lamination. 

