One can first define a 'proper algebraic space' $X,$ using its 'underlying space' $|X|,$ and then define a morphism of algebraic spaces $f: X \to Y$ to be proper if for any affine (or just quasi-compact) scheme mapping into $Y,$ the fiber product gives a proper algebraic space. Finally define an Artin stack $X$ to be proper separated if the diagonal (which is representable) to be is proper, i.e. for any algebraic space mapping into $X \times X,$ the fiber product....
One can also define a 'proper Artin stack' similarly. See Laumon and Moret-Bailly.
