Q: Why is topos theory not taken seriously?
The way this question is phrased seems too subjective, but I may point out that topos theory has been suggested as an alternative mathematical structure with which to formulate quantum theory. See the Bohr topos entry in nLab. It is not mainstream, but it is a serious development in the context of quantum logic.