Let $X = (X_1 \rightrightarrows X_0)$ be a category in schemes, such that the source and target maps are both smooth. By an argument using finite limits we can construct a subscheme $X_1^{iso} \subset X_1$ such that $X^{iso} = (X_1^{iso} \rightrightarrows X_0)$ is a groupoid in schemes, and this is universal in the sense that any internal functor $Y\to X$ from a groupoid $Y$ factors (strictly!) through $X^{iso}$.

My question is, can we show that the source and target maps of $X^{iso}$, which are composites $X_1^{iso} \hookrightarrow X_1 \to X_0$, are also smooth? If the inclusion of the scheme of invertible arrows was an open immersion I think we would be done.

I would also be happy if the proof only went through for algebraic spaces instead of schemes (this might be easier, who knows!).

This is a vast generalisation of the question I asked at M.SE a while back, which dealt with the case that the category $X$ was a monoid. User 'Matt E' showed it was true for the case that the monoid was smooth and of finite type over a field.