There are two concepts which both get called a topos, so it depends on who you ask. The more basic notion is that of an elementary topos, which can be characterized in several ways. The simple definition:
An elementary topos is a category C which has finite limits and power objects.
(A power object for A is an object P(A) such that morphisms B --> P(A) are in natural bijection with subobjects of A x B, so we could rephrase the condition "C has power objects" as "the functor Sub(A x -) is representable for every object A in C").
The issue with the simple definition is that it doesn't show you why these things are actually interesting. It turns out that a great deal follows from these axioms. For example, C also has finite colimits, exponential objects, has a representable limit-preserving functor P: C^op --> Doct where Doct the category of Heyting algebras such that if f: AxB --> A is the projection map for some objects A and B in C, then P(A) --> P(AxB) has both left and right adjoints considered as a morphism of Heyting algebras, etc etc. What the long-winded definition boils down to is "an elementary topos the the category of types in some world of intuitionistic logic." There's an incredible amount of material here; the best place to start is probably MacLane and Moerdijk's Sheaves in Geometry and Logic. The main reference work is Johnstone's as-yet-unfinished Sketches of an Elephant, but I certainly wouldn't start there.
The other major notion of topos is that of a Grothendieck topos, which is the category of sheaves of sets on some site (a site is a (decently nice) category with a structure called a Grothendieck topology which generalizes the notion of "open cover" in the category of open sets in a topological space). Grothendieck topoi are elementary topoi, but the converse is not true; Giraud's Theorem classifies precisely the conditions needed for an elementary topos to be a Grothendieck topos. Depending on your point of view, you might also look at Sheaves in Geometry and Logic for more info, or you might check out Grothendieck's SGA4 for the algebraic geometry take on things.
