Why is a topology made up of 'open' sets? I'm ashamed to admit it, but I don't think I've ever been able to genuinely motivate the definition of a topological space in an undergraduate course. Clearly, the definition distills the essence of many examples, but it's never been obvious to me how it came about, compared, for example, to the rather intuitive definition of a metric space. In some ways, the sparseness of the definition is startling as it tries to capture, apparently successfully, the barest notion of 'space' imaginable.
I can try to make this question more precise if necessary, but I'd prefer to leave it slightly vague, and hope that someone who has discussed this successfully in a first course, perhaps using a better understanding of history, might be able to help me out.
Added 24 March: 
I'm grateful to everyone for their thoughtful answers so far. I'll have to think over them a bit before I can get a sense of the 'right' answer for myself. In the meanwhile, I thought I'd emphasize again the obvious fact that the standard concise definition has been tremendously successful. For example, when you classify two-manifolds with it, you get equivalence classes that agree exactly with intuition. Then in as divergent a direction as the study of equations over finite fields, there is the etale topology*, which explains very clearly surprising and intricate patterns in the behaviour of solution sets. 
*If someone objects that the etale topology goes beyond the usual definition, I would argue that the logical essence is the same. It is notable that the standard definition admits such a generalization so naturally, whereas some of the others do not. (At least not in any obvious way.) 
For those who haven't encountered one before, a Grothendieck topology just replaces subsets of a set $X$ by  maps $$Y\rightarrow X.$$ The collection of maps that defines the topology on $X$ is required to satisfy some obvious axioms generalizing the usual ones.
Added 25 March: 
I hope people aren't too annoyed if I admit I don't quite see a satisfactory answer yet. But thank you for all your efforts. Even though Sigfpe's answer is undoubtedly interesting, invoking the notion of measurement, even a fuzzy one, just doesn't seem to be the best approach. As Qiaochu has pointed out, a topological space is genuinely supposed to be more general than a metric space. If we leave aside the pedagogical issue for a moment and speak as working mathematicians, a general concept is most naturally justified in terms of its consequences. As pointed out earlier, topologies that have no trace of a metric interpretation have been consequential indeed.
When topologies were naturally generalized by Grothendieck, a good deal of emphasis was put on the notion of an open covering, and not just the open sets themselves. I wonder if this was true for Hausdorff as well. (Thanks for the historical information, Donu!) We can see the reason as we visualize a two-manifold. Any sufficiently fine open covering captures a combinatorial skeleton of the space by way of the intersections. Note that this is not true for a closed covering. In fact, I'm not sure what a sensible condition might be on a closed covering of a reasonable space that would allow us to compute homology with it. (Other than just saying they have to be the simplices of a triangulation. Which also reminds me to point out that homology can be computed for ordinary objects without any notion of topology.)
To summarize, a topology relates to analysis with its emphasis on functions and their continuity, and to metric geometry, with its measurements and distances. However, it also interpolates between these and something like combinatorial geometry, where continuous functions and measurements play very minor roles indeed.
For myself, I'm still confused.
Another afterthought: I see what I was trying to say above is that open sets in topology provide an abstract framework for describing local properties of functions. However, an open cover is also able to encode global properties of spaces. It seems the finite intersection property is important for this, but I'm not able to say for sure. And then, when I try to return to the pedagogical question with all this, I'm totally at a loss. There are very few basic concepts that trouble me as much in the classroom.
 A: Hello Minhyong. I think other people have already given several excellent answers about how
to motivate topology, and I'm not sure I have much to add. But there are a couple of 
other parts to your question. I had the impression that the notion of a topological
space was introduced by Hausdorff in his 1914 book on set theory (Mengenlehre). But my history is somewhat shaky, and it would be nice if someone else could confirm this.
It certainly seems that by the early 20th century there was a radical shift in the way
mathematics was done.
Jumping ahead to your added comment, I agree that a Grothendieck topology is a very
natural extension of this idea. However, for Grothendieck this was secondary;
the associated category of sheaves or topos was the important thing. Of course, you know
that, but perhaps not everyone does.
A: I would say that topology is defined in terms of open sets and closed sets.  I think it is motivated by two theorems from the Calculus.  The Bolzano-Weierstrass  theorem and the intermediate value theorem.  
In its simplest form, the Bolzano-Weierstrass theorem says that an infinite subset of a closed bounded interval $[a,b]$ of the real numbers has a limit point.  You find that limit point as follows.
As there are infinitely many points in the set, then there are infinitely many in the left half or the right half of $[a,b]$. Say the left half. Divide that interval in half and there are infinitely many in one half or the other. Proceed this way to produce a sequence of closed intervals
$I_{n+1}\subset I_n$ with the length of the $n$th interval equal to $\frac{b-a}{2^n}$. By Cantor's theorem $\cap_n I_n$ is nonempty, and the point in there is your limit point.
It didn't really make a difference if you broke the interval into two pieces or 10 pieces.  This leads to the notion of compactness, by saying that every open cover has a finite subcover.
The fact that the intersection of the closed intervals $I_n$ is nonempty is the complementary
notion that a collection of closed subsets with the finite intersection property has nonempty intersection.  The proof of the Bolzano-Weierstrass theorem leads you to think of open and closed sets. 
A similar analysis of the proof of the intermediate value theorem leads likewise to open sets and closed sets.
Really, the concept of a topology was an incredible creative leap, that allowed people to take ideas from the Calculus and apply them in other places.  Similar leaps to me, are the notion of sigma algebra, distribution (in the PDE sense), and the construction of homological algebra.  :)
A: This may be a naive answer, but for me the concept captured by a topological space is when a point is infinitely close to a set. This happens when the point is in the closure of the set (or, equivalently, when every neighborhood of the point intersects the set). The definition by open sets may obscure this. 
A: I don't think that Grothendieck topologies should be viewed as analogous to ordinary topologies.  It is true that a topology on a set induces various Grothendieck topologies on various categories, but so does every system of basic open neighborhoods.  In my opinion, it is more fruitful to think of a Grothendieck topology as the analogue of a system of basic open neighborhoods and a topos as the analogue of a topological space.
Let me try to answer the following question, which may or may not be the question that is actually being asked: Why do we prefer topological spaces to systems of basic open neighborhoods, and topoi to Grothendieck topologies?  I think that the answer has to do with morphisms.
To give a morphism of spaces with basic open neighborhoods, one must give a function that respects those neighborhoods.  One can't, however, require that the pre-image of a basic open neighborhood be open.  Instead, one has to require that each point contained in the pre-image of a basic open neighborhood have a basic open neighborhood inside the pre-image.
Not only is this definition more complicated than the one for topological spaces (and the extension to Grothendieck topologies is by no means obvious!), but there are multiple distinct but isomorphic systems of open neighborhoods on the same space.  A topology is a maximal system of open neighborhoods in a given isomorphism class, which makes it a "best model" for a particular notion of nearness.
Another interpretation that makes this model appealing is that given a system of basic open neighborhoods, the open sets are the "local properties" (those that hold at a point if and only if they hold at all sufficiently nearby points).  (If one believes a slogan like "there are two -1-categories: TRUE and FALSE" then open sets are "-1-sheaves"; this completes the formal analogy with Grothendieck topologies and their associated topoi, which are their associated categories of "0-sheaves".)
A: At least for me, the first time I learned about a metric space, was to discuss when sequences converge. I am not a math-history buff, but the concept of metric seems to stem from the need to formalize the concept of convergence. However, everything about of convergence depends only on the topology the metric generates. Hence one only needs to understand the open sets.
Topological spaces generalize metric spaces in the sense that every metric space gives rise to one and all concepts of convergence are captured by this topological space. Even more, the category of metric spaces and continuous maps sits inside the category of topological spaces as a full subcategory (this is just saying that a map between metric spaces is continuous if and only if the preimage of an open is open). However, you may object, and justifiably, that you can embed metric spaces into several categories, so, why is topological spaces the right generalization?
There are many "practical answers", for instance, the wide range of examples of abstract spaces which are not metric spaces, e.g., Spec(R) of a ring. However, the core of the matter is that topological spaces correctly axiomatize the notion of convergence. What I mean by this is, a topological space is completely determined by the convergence of the ultrafilters on its underlying set. This is seem most notably for compact Hausdorff spaces; a topological space is compact Hausdorff if and only iff every ultrafilter has a unique limit, and in fact, compact Hausdorff spaces are precisely the algebras of the ultrafilter monad. However, we are interested in non-compact examples, since we study unbounded metric spaces for example- but even here we can have ultrafilters with no limit- so, a complete generalization should take this into account. Furthermore, the space Spec(R) is very often non-Hasudorff, which means, ultrafilters which have a limit point, may have more than one. So, to understand convergence is to understand the set of limits of each ultrafilter. If X is a space and BX is its set of ultrafilters, we get a map BX->P(X) which sends each ultrafilter to its set of limit points (possibly empty). This corresponds to a relation $R \subset X \times BX$, which made be seen as a map BX->X in the bicategory of sets and relations. More precisely, we get a "relational algebra" for the ultrafilter monad. The converse is true as well: the category of topological spaces is equivalent to the category of relational algebras for the ultrafilter monad. This a theorem of Barr. The upshot is, there is a bijection between topologies on a set and "convergence systems for ultrafilters" on that set.
Anyhow, this probably goes way beyond what you can explain to most undergraduates.
A: I have really quite enjoyed reading this thread, although I must confess I don't quite understand all of it. It has been many years since I have studied topology of any sort, and not only am I rusty, my mind is aging, and not as agile as it was in my youth. Anyway, for what it's worth, here is my take on basic definitions of topology.
My first introduction to formal topology (rather than the specialized versions of it in real and complex analysis) was in simple homotopy theory, and the textbook I used gave the open sets axiomatically as a starting point. Although I saw (with some difficulty) that this was a generalization of the properties of the real vector spaces I was used to, and although it was in an algebraic style, I struggled with it's non-intuitive presentation.
I believe (although I am hardly an expert) that "nearness" is a better approach to topology. It makes sense, even to a non-mathematician. For example, the statement of continuity in such a setting is simplicity itself: f:D->R is continuous at a point x in D, when: x is near a set A implies f(x) is near f(A). Furthermore, the axiomatic presentation of neighborhoods is simpler than the axioms for open sets in one important respect, you only need to consider the meet (for sets, intersection) of two neighborhoods, and the (possibly partial) order of the collection of the neighborhoods (for sets, the natural ordering on the power set, containment).
In a metric space, which automatically comes with a rich topological structure, you can easily resolve these notions to the traditional definitions. You lose the purely geometric flavor of topology in the process, though. Topology is concerned with matters in the small, and in the large, the nature of what are defined to be neighborhoods to a large part determines how intricate the spatial structure is.
As far as the aesthetic of why unions can be infinite, but intersections have to be finite, I have 2 thoughts: first, open and closed sets are somewhat dual notions, it is entirely possible to begin with the notion of a closed set, in which case you can allow only finite unions, but infinite intersections. In fact, this approach makes more sense for practical applications, since our physical tools for dealing with calculations (and our brains) are in fact finite. The second thought I have, is that when you take the union of two sets, you always get something "bigger", but when you take the intersection, you may get a null result. It seems natural to restrict the basic study to finite intersections, because infinite intersections can behave qualitatively different than infinte unions (similar to how, in whole numbers, substraction isn't always possible, but addition is).
Getting extra stuff is quite common in mathematics, you extend a field, or create a semi-direct product, or consider generated objects. But all this "extra stuff" is rather meaningless without some core thing that has some intrinsic behavior. In topology, I believe nearness should be that core thing. Much of topology's development was motivated by the idea of getting a handle on what a limit (and convergence) ought to mean, and these are notions which have their roots in approximation.
So, naively, one could argue, in real (one-dimensional) analysis we use open intervals as the basic building block, since we are often concerned about local behavior on very small intervals, and using that to extend to larger sets. extending this to a collection of open sets, or to a collection of neighborhoods can be shown to be equivalent constructions; however, using open sets focuses more on the "boundarylessness" of these sets (and thus emphasizes the density of the real numbers), whereas using neighborhoods focuses more on the "localness" of these intervals, lending itself more easily to abstraction while keeping some intuitive spatial idea.
A: It is indeed appropriate to ask "why open sets in topology ?"
However, the answer is not so simple precisely since the concept of topology proves to be far more deep and complex than Hausdorff, Kuratowski or Bourbaki have ever imagined, and which may call the HKB topology.
For starters, it turned out decades ago that the usual, open set based, that is, HKB topology leads to a category which is not Cartesian closed. And this creates serious difficulties when dealing with topologies on function spaces, and in particular, in duality theory for locally convex spaces.
More simply, and without categories, there are most important topological type processes in mathematics which simply cannot be described by HKB topologies. Such are, for instance, in measure theory and partially ordered spaces. 
As a consequence, various more general concepts of pseudo-topologies have been suggested. What happened, however, is that the doors thus opened up proved to be too large for those who tried to pursue them, or would have liked to use them ...
In other words, topologies beyond HKB are a far less cozy venture than usually customary in mathematics ...
Some details about the above and relevant references may be found in   
arXiv:1001.1866 [pdf, ps, other]
Title: Beyond Topologies, Part I 
Authors: Elemer E Rosinger, Jan Harm van der Walt 
Subjects: General Mathematics (math.GM) 
arXiv:1005.1243 [pdf, ps, other]
Title: Rigid and Non-Rigid Mathematical Theories: the Ring $\mathbb{Z}$ Is Nearly Rigid 
Authors: Elemer E. Rosinger 
Subjects: General Mathematics (math.GM) 
A: I want to complete something said by Andrew Stacey above: like him I think that the only reason to motivate the use of the open sets it's because they are more easy to use. Topology is the study of property preserved by invertible continuous transformation (following the Erlangen program): this definition clearly need the notion of continuity, I've always thought of continuity as the relation of proximity of points, so the first thing to do topology is to define the notion of proximity and neighbourhood are most natural way to do so (at least for me). Anyway dealing with neighbourhoods is more complex than working with open set, for example the definition of topology with neighbourhoods require five axioms while classical definition with open sets require just three axioms. So while it seems more natural the study of topology via neighbourhoods it is more convenient dealing with open sets which allow to simplify the proofs.
I hope this answer my help.
A: To me, the concept of an open set is a distillation and abstraction of the properties of open intervals (on the real line) that are critical to defining and working with a continuous function. In my opinion students should never be introduced to the abstract concepts of topology (notably, the concepts of open sets and compactness) unless they have already mastered analysis of functions on the real line and finite-dimensional vector spaces and understand thoroughly the role of open sets and compactness in those settings.
A: I have long understood that the initial ideas of  topology arose from the notion of "neighbourhood" and were then found to be equivalent to the definition in terms of open sets. One advantage of the neighbourhood concept was that the definition of continuity using that is  nearer to the $\varepsilon$-$\delta$ definition used in analysis. 
The neighbourhood definition is more easily motivated than that in terms of open sets, but one then shows the equivalence. However one finds difficulties with the neighbourhood definition in defining, say,  identification spaces, and this illustrates nicely a feature of mathematics, that equivalent concepts may have their best uses in different areas. Horses for courses! 
Einstein wrote in 1915: 
"Concepts which have proved useful for ordering things easily assume so great an authority over us, that we forget their terrestrial origin and accept them as unalterable facts. They then become labelled as conceptual necessities, a priori situations, etc. The road of scientific progress is frequently blocked for long periods by such errors. It is therefore not just an idle game to exercise our ability to analyse familiar concepts, and to demonstrate the conditions on which their justification and usefulness depend, and the way in which these developed, little by little... "
Thus Grothendieck in his 1984 "Esquisse d'un programme" Section 5, argues that the notion of topological space is motivated from analysis rather than geometry, and the latter requires spaces with more structure, in particular what he calls stratified spaces. I have found filtered spaces important in basic homotopical algebraic topology. 
A: It might be a futile attempt to add anything worthwhile to this long list of interesting answers, but let me add my own pedestrian $0.02:
One may think of topology as a set of rules about what's close to what. In other words, it tells me that if I pick a point in the space, then there are several rules (i.e., open sets) that tell me that with respect to some question this set of points is "close" to my chosen point. Considering many rules (i.e., intersecting the open sets) gives me better and better approximation of which points are "really close" to the chosen one. It seems clear that then the union and intersection of these rules would have to belong to the rules.
If we were in a Euclidean space, then we might agree that one way to measure what's close is to put a small (open) ball around a point. If we can't measure, we can't do this, so we need to do something more general and a single open set will not be enough (it's not enough even in a Euclidean space as the radius of the ball that defines "closeness" would certainly depend on the way we want to measure closeness).
So far both open and closed sets would qualify for these rules, but I feel that open sets work better: A rule of "closeness" should be independent of any single point. In other words, a rule should behave the same with respect to any point it applies to (i.e., any point contained in the corresponding set). This clearly picks open sets over closed sets. 
I suppose one might say that none of this explains what happens with infinitely many rules/sets. I suppose we could say that if we take an infinite set of rules that define closeness, then on one hand we might still say that satisfying any one of the rules is still a reasonable rule while satisfying all the rules is a little bit too much to ask. If you feel this part of my argument is a little shaky, then we agree. I don't have a very good explanation for the behavior of infinite unions and intersections. If I was indeed trying to explain this to undergrads, then at this point I would probably switch over to see what happens in a Euclidean space with all this non-sense about rules of "closeness" and come to the conclusion that a good way to define rules is to say that their corresponding sets contain little balls around every point in them. Then deduce the axioms of open sets in a topology and then say that we should see what these give us if we forget that we were in a Euclidean space. 
A: Here's how I like to think about it. 
We can all agree that a topological space should be a set $X$ together with some extra structure encoding how the points of $X$ fit together. It seems pretty reasonable ask that this structure is sophisticated enough to answer the following question whenever $x \in S \subset X$:

Given any choice of "direction" is there freedom to nudge $x$ some small "amount" in that direction without bumping into any points of $X \setminus S$?

We say that $x$ is an interior point of $S$ if the answer to the above question is "yes". I would say the following assertions about interior points are completely reasonable.


*

*Any $x \in X$ is an interior point of $X$.

*If $S \subset T \subset X$ and $x$ is an interior point of $S$, then $x$ is an interior point of $T$.

*If $S,T \subset X$ and $x$ is an interior point of both $S$ and $T$, then $x$ is an interior point of $S \cap T$.


For instance, (1) holds because there are no points in $X \setminus X$ to concern ourselves about bumping into. (3) holds because, if I specify a direction, then I can move $x$ an amount $a_s$ (in this direction) without hitting points from $X \setminus S$ and an amount $a_t$ without hitting points from $X \setminus T$, so if I move $x$ the smaller of these two amounts, I won't hit anything in $X \setminus (S \cap T)$.
If we take the above as axioms for a machine that tells us which points are interior to which sets, and then define an open set to be a set each of whose points is an interior point, then it is simple to recover the standard axioms for open sets:


*

*$\varnothing$ and $X$ are open.

*The union of arbitrarily many open sets is open.

*The intersection of two open sets is open.


The only issue I can see with this approach is that one might be able to convince oneself that interior points should satisfy more axioms. For instance, if $X = \{0,1\}$ and $1$ can be moved a little bit in any direction without bumping into $0$, then shouldn't it be possible to move $0$ a little bit in any direction without bumping into $1$? This would seem to preclude the existence of the Sierpiński topology $\{\varnothing, \{1\} ,X\}$. Or perhaps this is merely an invitation to be more imaginitive about the geometry of the situation? For instance, maybe there is a little round bowl with $1$ at the bottom and $0$ is sitting on the rim. If I give a $0$ a little push in the direction of $1$, no matter how small, $0$ will roll into the bowl and bump into $1$.
A: After reading the comments on Sigfpe's answer I realized that it would be useful to make a rigorous argument to explain why "rulers" or as like to call them "observable properties" should be open sets. In the process I'd like to explain how we can view general topology as an idealized version of computation by interpreting topological spaces as data types and continuous maps as computable functions.
Computationally an observable property $P$ of a data type $A$ corresponds to a semi-decision procedure. In other words a computable function $\chi_P: A \to Unit$ which returns the unique value $()$ of type $Unit$ if $a \in A$ has the property $P$ and runs forever otherwise. We can interpret $P$ as a subset of $A$ and $\chi_P$ as it's characteristic function. Clearly observable properties pull back under computable functions since if $f:B \to A$ is a computable function $\chi_P \circ f$ is a semi-decision procedure.
Let's translate this into topological language. If we interpret $A$ and $B$ as topological spaces and $f:B \to A$ as a continuous map we have that $f^{-1}(P)$ is observable if $P$ is. Thus it makes sense to interpret observable properties as open sets. We can make this correspondence more precise if we notice that every open set $P$ in $A$ corresponds to a map $\chi_P: A \to \mathbb{S}$ where $\mathbb{S}$ is the Sierpinski space. Thus in our translation $Unit$ corresponds to $\mathbb{S}$, the open point of $\mathbb{S}$ corresponds to $()$, and the closed point $\bot$ of $\mathbb{S}$ corresponds to nontermination.
Now a question remains: why did we choose to represent observable properties by open sets instead of closed sets? The answer lies in the way observable properties behave under intersection and union. Let $P$ and $Q$ be observable properties. The intersection $P \cap Q$ is an observable property we can write a semi-decision procedure $\chi_{P \cap Q}$ by running $\chi_P$ and $\chi_Q$ in succession. Similarly notice that $P \cup Q$ is observable since we can write a semi-decision procedure $\chi_{P \cup Q}$ that runs $\chi_P$ and $\chi_Q$ in parallel and outputs $()$ if one of $\chi_P$ and $\chi_Q$ does. If you have an infinite number of computers it is clear that you can generalize this construction to an infinite union $\bigcup_{i \in I} P_i$ by running all the $\chi_{P_i}$ in parallel. However this will not work for an infinite intersection $\bigcap_{n \in \mathbb{N}} P_n$ because if $\chi_{P_n}$ takes $n$ seconds to terminate, then even running all the $\chi_{P_n}$ in parallel will not help.
I can't help but list out a few other things to ponder in light of this dictionary:

*


*

*A space $X$ is discrete if $=:X\times X \to \mathbb{S}$ is continuous  

*A space $X$ is Hausdorff if $\neq: X\times X \to \mathbb{S}$ is continuous  

*A space $X$ is compact if the map $\forall_X: (X \to \mathbb{S}) \to \mathbb{S}$ is continuous  

*An observable property $P$ of $T$ is decidable if and only if $P$ is clopen  

*On a sequential machine we can write a semi-decision procedure for a countable union of observable properties but not an uncountable union. Does this say anything about topology?

Here are some nice references:




*Alex Simpson - Topological Spaces from a Computational Perspective

*Martin Escardo - Synthetic Topology of Data Types and Classical Spaces

A: One of the comments on the original post said that you can define a topology in terms of neighbourhoods. I'd like to amplify on that comment because it's the answer I favour too, if you want to do things as intuitively as possible. In fact, you can do it with basic open neighbourhoods, which is often nicer, for reasons I'll come to in a moment.
The first step would be to axiomatize the notion of a basic open neighbourhood. So it would consist of properties like that if N is a b.o.n. of x then x is an element of N, that if y is also an element of N then there is a b.o.n. N' of y such that N' is a subset of N (and in many systems one would be able to take N'=N), that the intersection of two b.o.n.s of x is another one, and so on. Suppose we've got all that sorted out. Then the rest of the definitions are just like metric space definitions without the need to reformulate those definitions in terms of open sets. To give the most important example, a function $f:X\to Y$ is continuous at x if and only if the following condition holds: for every b.o.n. M of f(x) there exists a b.o.n. N of x such that $f(N)\subset M$. Of course, in a metric space the basic open neighbourhoods of x are the open balls $B_\epsilon(x)$. 
In the usual definition of continuity for maps between topological spaces, one never talks about continuity at a point, but it is perfectly possible and natural to do so, as the above shows.
Here's another example: a set F is closed if and only if for every x not in F you can find a basic open neighbourhood N of x that is disjoint from F. Oh, and I should have said that a set U is open if and only if for every x in U you can find a basic open neighbourhood N of x such that $N\subset U$. 
The one thing you can't do is reformulate these definitions in terms of sequences, for the simple reason that the sequence reformulations do not generalize to topological spaces (unless you replace them by nets).
Added later: I've just seen some more of the comments on the original post. Much of what I have said is implicit in those comments, but perhaps it is useful to have it spelt out.
A: First, note that a mapping between metric spaces is continuous if and only if the inverse image of an open set is always open. There are various concepts for metric spaces that you can likewise find equivalent formulations for in terms of open (and closed) sets, for example compactness. Convergence of a sequence to a point can be rephrased in terms of neighbourhoods of the point, with no reference to any ε. Then you could, for example, notice how you can talk about pointwise convergence of functions, but there is no corresponding metric. So you need a more general framework for talking about different kinds of convergence, and soon enough, topological spaces won't seem so strange anymore.
A: Disclaimer: There are many topologists here and they may not like the philosophical flavour of my answer :-) 
I think it all starts with the end... actually with the notion of "end". Take an open interval (a,b). It is bounded, yet you cannot reach its ends! At first this may look weird, but then one realizes this weirdness is to be attributed to the mathematically exact observation of this object (mathematicians can distinguish between so many things like point, set of points, boundedness, boundary etc.). Encountering such "weirdness" only shows the strong need for an exact and abstract formulation of "having no end". The lack of ends we then call "openness". If we are looking for the best generalization of this concept, the first approach would be of course a set-theoretical one. And in fact, in the case of intervals it turns out that the property "having no ends" is inhereted into arbitrary unions and finite intersections. Any attemps to expand this lead either to contradictions to our basic example or to unjustified reduction in generality. 
A: My answer will not be of philosophical nature, neither historical, but perhaps pedagogical.
I find Munkress' Topology a great book. Among other merits, because its introduction, which I summarize as follows:


*

*You recall what a metric space is. Define open balls and subsequently, open sets. Prove that, in a metric space:
1.1 The empty set and the total space are open sets.
1.2 The union of an arbitrary number of open sets is an open set.
1.3 The intersection of a finite number of open sets is an open set.

*Recall what a continuous map between metric spaces is (the $\epsilon$-$\delta$ definition). Prove the theorem that says that a map between metric spaces $f: X \longrightarrow Y $ is continuous if and only if $f^{-1} (U) \subset X$ is an open set for every open set $U \subset Y$.
And you have a motivation for the definition of topological space and continuous map as well.
Of course this is not an historical explanation of how topological spaces arised, nor does it justify why you chose these properties of open sets in metric spaces and not others: "experience" has told us that these are the good ones. (For instance, if I'm not wrong, when Hausdorff first defined topological spaces included the property of being... Hausdorff among the axioms. "Experience" -and not an a priori argument- showed us that it could be interesting to work with non-Hausdorff topological spaces.)
A: Think of the half-open interval $(0,1]$ with the usual open sets (e.g. $(1-\varepsilon,1]$ is an open neighborhood of 1.  Then modify the collection of sets considered "open" so that every open neighborhood of 1 contains some set of the form $(1-\varepsilon,1] \cup (0,\varepsilon)$.  See if students understand that this modification in which sets are considered open also modifies the way in which the space is connected together.
A: I'm going to take a finite, "combinatorial geometry" approach, thinking about, for instance, convex polytopes.
Suppose we have a finite set of things. For instance, a tetrahedron is made up of 4 points, 6 lines, 4 faces, and one solid. We notice, first, that some of these things are adjacent to each other, and some things are not adjacent to each other. So we have a graph.
Then we further note that this graph can be directed. If two things are adjacent, one of them must be bigger. If they have the same dimension, we have forgotten about the boundary that, in fact, separates them. This is a consequence of including, in some sense, all the shapes you can, to maximally clarify the geometry of your space.
So we have a directed graph. It is easy to verify that this graph should satisfy the axioms of a partially ordered set. If $A \geq B$ ($B$ is an edge of $A$) and $B \geq C$ ($C$ is a vertex of $B$) then $A\geq C$ ($C$ is a vertex of $A$). 
Now, finite partially-ordered sets are just finite T0 topological spaces. The open sets contain everything $\geq$ their elements, and the closed sets contain everything $\leq$ their elements. It is not obvious in this context why these are natural objects of study, although they are fairly easy to define, and thus must be useful for something.
To determine the difference between finite and infinite union, we must break out of the finite world. We're going to do that, however, only by breaking it up into smaller finite pieces. A face might turn into 4 faces, 4 edges, and 1 vertex, for instance. A set on the unbroken space becomes a set on the broken space. Its closure is preserved, while the smallest open set containing it gets smaller.
Thus, though in the finite case we can consider the smallest closed set containing something or the smallest open set containing it, only the first notion is preserved as we increase the number of objects in our space and decrease their size, bringing us closer to infinite, continuous mathematics. Thus, we abandon the notion of a smallest open set containing something, which means we must abandon infinite intersection of open spaces, and therefore, infinite union of closed spaces.
A: Topological spaces are good abstract spaces to study limit and continuity, just as vector spaces are good abstract spaces to study linear combinations.  Like many abstractions, proofs are studied in less abstract settings (e.g., $\mathbb{R}$) to see what makes them tick.
So why open sets?
Topology is defined in terms of open sets because that formulation was introduced (by Hausdorff?) at just the right time to become popular and drive out any competing formulations.  There are quite a few equivalent formulations: closed sets; neighborhoods; operation of taking interiors; closure operation; predicate that says when a point is a limit point of a set; and so forth.
A: Topology is the art of reasoning about imprecise measurements, in a sense I'll try to make precise.
In a perfect world you could imagine rulers that measure lengths exactly. If you wanted to prove that an object had a length of $l$ you could grab your ruler marked $l$, hold it up next to the object, and demonstrate that they are the same length.
In an imperfect world however you have rulers with tolerance. Associated to any ruler is a set $U$ with the property that if your length $l$ lies in $U$, the ruler can tell you it does. Call such a ruler $R_U$.
Given two rulers $R_U$ and $R_V$ you can easily prove a length lies in $U\cup V$. You just hold both rulers up to the length and the length is in $U\cup V$ if one or the other ruler shows a positive match. You can think of $R_{U\cup V}$ as being a kind of virtual ruler.
Similarly you can easily prove that a point lies in $U\cap V$ using two rulers.
If you have an infinite family of rulers, $R_{U_i}$, then you can also prove that a length lies in $\bigcup_i U_i$. The length must lie in one of the $U_i$ and you simply exhibit the ruler $R_{U_i}$ matching for the appropriate $i$.
But you can't always do the same for $\bigcap_i U_i$. To do so might require an infinitely long proof showing that all of the $R_{U_i}$ match your length.
A topology is a (generalised) set of rulers that fits this description.
Your notion of 'measurement' in whatever problem you have might not match the notion that the above description tries to capture. But to the extent that it does, topology will work as a way to reason about your problem.
A: Not sure if anyone has mentioned this article, but Gregory Moore discussed the development of the notion of open sets vs other historical approaches, in the paper "The emergence of open sets, closed sets, and limit points in analysis and topology" in Historia Mathematica, no. 35, 2008, pages 220-241. Makes for an interesting read.
A: This is an attempt to synthesize ideas that have appeared in other answers, for example sigfpe's and Tim Perutz's.  Feel free to edit if you think the ideas can be better expressed.
The idea I want to back is that a topological space is an environment $X$ in which the notion of checking the truth of a statement locally makes sense.  In the actual language of topological spaces, we want to be able to talk about statements which are true for a space $X$ if and only if they're true for every open set in an open cover of $X$, and the same should be true for every subspace of $X$.  (For example, continuity and differentiability of a function both have this property.)  
But whatever an open cover is, it should consist of elements chosen from a distinguished collection of subsets $\mathcal{P}$ of $X$ having certain properties.  The empty set and $X$ should both be in $\mathcal{P}$ because checking a statement about $X$ is trivially equivalent to checking it on $X$ and on the empty set.  $\mathcal{P}$ should be closed under arbitrary unions because a collection of open sets automatically forms an open cover of its union.  $\mathcal{P}$ should be closed under binary intersections because one should be able to build an open cover of a subspace $S$ of $X$ by intersecting an open cover of $X$ with $S$, and if $S$ is itself open, an open cover of $S$ should be extendable to an open cover of $X$.  
I don't think I've explained myself very well, though.  I also wish I knew enough to say something about the relationship between topology and logic that the above seems to suggest.  But one reason I like this perspective is that it suggests certain definitions naturally, such as the definition of compactness or of a manifold.

Some soapboxing: while I can see the pedagogical value of thinking about topological spaces as a natural generalization of metric spaces or even just of $\mathbb{R}$, I think the idea of a topological space is deeper than these roots suggest and I think Minhyong is looking for an answer that reflects this.  In other words, I am of the opinion that the definition of a topological space is more natural than the definition of a metric space (or even of $\mathbb{R}$!), so one shouldn't use the latter to motivate the former.  But this is just an opinion.
A: The textbook presentation of a topology as a collection of open sets is primarily an artefact of the preference for minimalism in the standard foundations of the basic structures of  mathematics.  This minimalism is a good thing when it comes to analysing or creating such structures, but gets in the way of motivating the foundational definitions of such structures, and can also cause conceptual difficulties when trying to generalise these structures.
An analogy is with Riemannian geometry.  The standard, minimalist definition of a Riemannian manifold is a manifold $M$ together with a symmetric positive definite bilinear form $g$ - the metric tensor.  There are of course many other important foundational concepts in Riemannian geometry, such as length, angle, volume, distance, isometries, the Levi-Civita connection, and curvature - but it just so happens that they can all be described in terms of the metric tensor $g$, so we omit the other concepts from the standard minimalist definition, viewing them as derived concepts instead.  But from a conceptual point of view, it may be better to think of a Riemannian manifold as being an entire package of a half-dozen closely inter-related geometric structures, with the metric tensor merely being a canonical generating element of the package.
Similarly, a topology is really a package of several different structures: the notion of openness, the notion of closedness, the notion of neighbourhoods, the notion of convergence, the notion of continuity, the notion of a homeomorphism, the notion of a homotopy, and so forth.  They are all important, and it is somewhat artificial to try to designate one of them as being more "fundamental" than the other.  But the notion of openness happens to generate all the other notions, and has a particularly elegant and simple axiomatisation, so we have elected to make it the basis for the standard minimalist definition of a topology.  But it is important to realise that this is by no means the only way to define a topology, and adopting a more package-oriented point of view can be preferable in some cases (for instance, when generalising the notion of a topology to more abstract structures, such as topoi, in which open sets no longer are the most convenient foundation to begin with).
A: I risk reviving a settled matter.
I aim these sketchy remarks at the expert teacher - not at the neophyte student. 
A motivation for open sets in topology might begin with a critique of measurement.  Though we often think of measurement in continuous terms, practical measurement really always comes down essentially to answering Boolean questions.  Thus the real-valued distance function $d(x,y)$ on a metric space carries the same information as a Boolean-valued function $D(x,y,r)$ where $D(x,y,r)=1$ iff $d(x,y)\leq r$. The Dedekind construction of the reals reflects the sort of divide-and-conquer process of actual measurement, the sort of process that takes us from $D$ to $d$, but not in finite terms.
Now in a non-metric topological space you just allow yourself a richer set of questions than you can index with a variable $r$ running over the reals.
Indeed mathematicians often identity sets with properties - having such and such a property means belonging to the (sub)set of all elements (of a given set) that have that property. Then we can measure the proximity of two things by which properties (that we care about) they share.  
At a technical level, the previous paragraph  has this reflection: just as a metric allows you to embed a metric space in a product of copies of the positive reals, a topology allows you to embed a general space in a product of copies of the $2$-element Sierpiński space. 
Now a student might reasonably challenge the appearance of Sierpiński space in this fundamental role: why the asymmetry? why have just one open point?  if binary decisions lie at the heart of the story, why not take as fundamental the $2$-element space discrete? 
I say the choice of Sierpiński space mirrors an aspect of practical life.  For certain questions, observation may, on the negative side, supply a full refutation, but on the positive side only at best lend support and never full confirmation.  For example, we may discover after careful observation that two quantities are not equal, but often we can only amass evidence that they are equal pending more precise measurement.  Another example,
when we witness a demise we learn that something wasn't eternal, but observation can never confirm that something is eternal.
This concept of decidability motives the axiomatic closure properties of open sets.
Intuitively, an open property admits confirmation by a finite amount of evidence.
An arbitrary disjunction of open properties (a union) gets confirmed by confirming
any one of them and thus also requires only a finite amount of evidence.  But a 
conjunction of open properties requires confirming them all, so we must limit ourselves,
at least a priori, to finite conjunctions (intersections).
In summary, the Sierpiński space may seem like a curiosity, if not a monstrosity, but it captures the essence of topology.  We have open sets because we care about continuous maps to Sierpiński space, whether consciously or not.  We care about continuous maps to Sierpiński space because we care about properties whether they are decidable or not (in the sense of the intuitionists, not in the sense of Turing), i.e., whether or not they disconnect the universe of possibilities.  Accepting Sierpiński space commits you to accepting the subspaces of its self-products.  The real conceptual hurdle for the topological neophyte lies in contemplating the ubiquity of undecidable properties.
Grothendieck topologies fit very nicely into this point of view (as it leads to topos theory).  In essence, Grothendieck challenges the doctrine of identifying properties with subsets.  For Grothendieck, a given property may only become visible if one breaks
a symmetry or observes some distinction between objects that previously seemed identical.  Thus singling out a property might require taking a cover rather than only passing to a subset.  
A: In this answer I will combine ideas of sigfpe's answer, sigfpe's blog, the book by Vickers, Kevin's questions and Neel's answers adding nothing really new until the last four paragraphs, in which I'll attempt to settle things about the open vs. closed ruler affair. 

DISCLAIMER: I see that some of us are answering a question that is complementary of the original, since we are trying to motivate the structure of a topology, instead of adressing the question of which of the many equivalent ways to define a topology should be used, which is what the question literally asks for. In the topology course that I attended, it was given to us in the first class as an exercise to prove that a topology can be defined by its open sets, its neighbourhoods, its closure operator or its interior operator. We later saw that it can also be stated in terms of convergence of nets. Having made clear these equivalence of languages, its okay that anyone chooses for each exposition the language that seems more convinient without further discussion. However, I will mantain my non-answer since many readers have found the non-question interesting. 

Imagine there's a set X of things that have certain properties. For each subset of S there is the property of belonging to S, and in fact each property is the property of belonging to an adequate S. Also, there are ways to prove that things have properties.
Let T be the family of properties with the following trait: whenever a thing has the property, you can prove it. Let's call this properties affirmative (following Vickers).
For example, if you are a merchant, your products may have many properties but you only want to advertise exactly those properties that you can show. Or if you are a physicist, you may want to talk about properties that you can make evident by experiment. Or if you predicate mathematical properties about abstract objects, you may want to talk about things that you can prove.
It is clear that if an arbitrary family of properties is affirmative, the property of having at least one of the properties (think about the disjunction of the properties, or the union of the sets that satisfy them) is affirmative: if a thing has at least one of the properties, you can prove that it has at least one of the properties by proving that property that it has.
It is also clear that if there is a finite family of affirmative properties, the property of having all of them is affirmative. If a thing has all the properties, you produce proofs for each, one after the other (assuming that a finite concatenation of proofs is a proof).
For example, if we sell batteries, the property P(x)="x is rechargeable" can be proved by putting x in a charger until it is recharged, but the property Q(x)="x is ever-lasting" can't be proved. It's easy to see that the negation of an affirmative property is not necessarily an affirmative property. 
Let's say that the open sets are the sets whose characteristic property is affirmative. We see that the family T of open sets satisfies the axioms of a topology on X. Let's confuse each property with the set of things that satisfy it (and open with affirmative, union with disjunction, etc.).
Interior, neighbourhood and closure: If a property P is not affirmative, we can derive an affirmative property in a canonical way: let Q(x)="x certainly satisfies P". That is, a thing will have the property Q if it can be proved that it has the property P. It is clear that Q is affirmative and implies P. Also, Q is the union of the open sets contained in P. Then, it is the interior of P, which is the set of points for which Q is a neighbourhood. A neighbourhood of a point x is a set such that it can be proved that x belongs to it. The closure of P is the set of things that can't be proved not to satisfy P.
Axioms of separation: If T is not T0, there are x, y that can't be distinguished by proofs and if it is not T1, there are x, y such that x can't be distinguished from y (we can think that they are apparently identical batteries, but x is built in such a way that it will never overheat. So if it overheats, then it's y, but if it doesn't, you can't tell).
Base of a topology: Consider a family of experiments performable over a set X of objects. For each experiment E we know a set S of objects of X over which it yields a positive result (nothing is assumed about the outcome over objects that do not belong to S). If you consider the properties that can be proved by a finite sequence of experiments, the sets S are affirmative and the topology generated by them is the family of all the affirmative properties. 
Compactness: I don't know how to interpret it, but I think that some people know, and it would be nice if they posted it. (Searchable spaces?)
Measurements: A measurement in a set X is an experiment that can be performed on each element of X returning a result from a finite set of possible ones. It may be a function or not (it is not a function if there is at least one element for which the result is variable). The experiment is rendered useful if we know for each possible result r a set T_r of elements for which the experiment certainly renders r and/or a set F_r for which it certainly doesn't, so let's add this information to the definition of measurement. An example is the measurement of a length with a ruler. If the length corresponds exactly with a mark on the ruler, the experimenter will see it and inform it. If the length fits almost exactly, the experimenter may think that it fits a mark or may see that it doesn't. If the length clearly doesn't fit any mark (because he can see that it lies between two marks, or because the length is out of range), he will inform it. It is sufficient to study measurements that have only a positive outcome and a negative outcome, a set T for which the outcome is certainly positive and a set F for which the outcome is certainly negative.
Imprecise measurements on a metric space: If X is a metric space, we say that a measurement in X is imprecise if there isn't a sequence x_n contained in F that converges to a point x contained in T. Suppose that there is a set of imprecise measurements available to be performed on the metric space. Suppose that, at least, for each x in X we have experiments that reveal its identity with arbitrary precision, that is, for each e>0 there is an experiment that, when applied to a point y, yields positive if y=x and doesn't yield positive if d(y,x)>e. Combining these experiments we are allowed to prove things. What are the affirmative sets generated by this method of proof? Let S be a subset of X. If x is in the (metric) interior of S, then there is a ball of some radius e>0 centered at x and contained in S. It is easy to find an experiment that proves that x belongs to S. If x is in S but not in the interior (i.e, it is in the boundary), we don't have a procedure to prove that x is in S, since it would involve precise measurement. Therefore, the affirmative sets are those that coincide with its metric interior. So, the imprecise measurements of arbitrary precision induce the metric topology.
Experimental sciences: In an experimental science, you make a model that consists of a set of things that could conceivably happen, and then make a theory that states that the things that actually happen are the ones that have certain properties. Not all statements of this kind are completely meaningful, but only the refutative ones, that is, those that can be proved wrong if they are wrong. A statements is refutative iff its negation is affirmative. By applying the closure operator to a non refutative statement we obtain a statement that retains the same meaning of the original, and doesn't make any unmeaningful claim. 
An example from classical physics: Assume that the space-time W is the product of Euclidean space and an affine real line (time). It can be given the structure of a four-dimensional real normed space. Newton's first law of motion states that all the events of the trajectory of a free particle are collinear in space-time. To prove it false, we must find a free particle that incides in three non-collinear events. This is an open condition predicated over the space W^3 of 3-uples of events, since a small perturbation of a counterexample is also a counterexample. Assuming that imprecise measurements of arbitrary precision can be made, it is an affirmative property. I think that classical physicists, by assuming that these kind of measurements can be done, give exact laws like Newton's an affirmative set of situations in which the law is proved false. I also suspect (but this has more philosophical/physical than mathematical sense) that the mathematical properties of space-time (i.e. that it is a normed space over an Archimedean field) are deduced from the kind of experiments that can be done on it, so there could be a vicious circle in this explanation.
A: There are several interpretations of the original question, but one is, why focus on open sets rather than closed sets? I have an unusual answer.
Suppose you want to do constructive mathematics. (Don't ask me why, you just do.) So you abstract the properties of open and closed subsets from the real line. Then you see that open subsets are closed under arbitrary union but only finitary intersection, OK. Dually, you see that closed sets are closed under arbitrary intersection but … not under finitary union! For example, the union of $ [ 0 , 1 ] $ and $ [ 1 , 2 ] $ cannot be proved to be closed. (The closure of the union is $ [ 0 , 2 ] $, but to prove that the union itself is all of $ [ 0 , 2 ] $ requires the lesser limited principle of omniscience. Or less formally, there is no definite method to decide whether a number near $ 1 $ is in $ [ 0 , 1 ] $ or in $ [ 1 , 2 ] $.) So open sets are better behaved and naturally you prefer to axiomatise them.
But as you continue with constructive topology, more advanced things fail, such as the Tychonoff Theorem (which implies the axiom of choice and thus excluded middle). Then you learn that this stuff works in locale theory, so you abandon traditional topological spaces for locales. And here the duality between open and closed is restored; a locale's frame of opens can just as well be interpreted as a coframe of closeds, and only tradition tells us to do the first.  (In the locale of real numbers, the union of the closed sublocales $ [ 0 , 1 ] $ and $ [ 1 , 2 ] $ is the closed sublocale $ [ 0 , 2 ] $, and the thing that you can't prove constructively is that every point in this union belongs to at least one of its addends.)
So this answer only works in a very unusual frame of mind: setting off down an unusual path but not going all the way.
A: It may seem hard to add a new answer to all this, but here's mine.  How to motivate the open set garbage of topological spaces:

Answer: Don't.

There are many ideas in mathematics that can be easily derived from some real situation, and I would count approximation (ie limits), metric spaces, and neighbourhoods as among these.  I think that it is quite easy to motivate the neighbourhood definition of topological spaces, for example, by considering real world examples of needing approximations that can't be controlled by metrics (for example, if you always need your approximations to be greater than the true value).
But one can take this line too far and try to motivate everything in mathematics from real-world situations and this, I think, misses a great opportunity to teach something that all students of mathematics need to learn: that when something is presented to you in a particular way, you don't have to accept that viewpoint but can choose a different one more suited to what you want to do.
We try to teach them this with bases of vector spaces: don't use the basis given, use one that makes the matrix look nice (diagonal if possible!).
So here, we can present topological spaces as sets with lots of declared neighbouhoods satisfying certain simple, intuitive rules.  But they are hard to work with so instead we work with open sets (sets which are neighbourhoods of all their points) because it makes life easier.

I should qualify the above a little.  It's written as a counterpoint to all the previous replies which try to justify open sets based on some intuition.  I'm not saying that those are wrong - far from it - just that with something like this, one should think carefully about the message one is sending to the students about mathematics.  At some point, they have to learn that mathematics strives to be clear and elegant rather than intuitive and vague, and it's a good idea to do this with an example like topological spaces where we are still close to the intuition, rather than something like function spaces where intuition often takes a hike.
A: Here's one of my favorite axiomatizations of topology!

To make a set $X$ into a topological space, you introduce a relation, "touches," between the elements of $X$ and the subsets of $X$. This relation must have the following properties:
  
  
*
  
*No point touches the empty subset.
  
*If $x$ is an element of $A$, then $x$ touches $A$.
  
*If $x$ touches $A \cup B$, then $x$ touches $A$ or $x$ touches $B$.
  
*If $x$ touches $A$, and every element of $A$ touches $B$, then $x$ touches $B$.
  
  
  Here, $x$ is an arbitrary element of $X$, and $A$ and $B$ are arbitrary subsets of $X$.

The first three axioms agree very well with my intuitive concept of "touching," and I find the fourth one at least tolerable, if not totally self-evident. If you leave out the fourth axiom, you get the definition of a pretopological space (a set with a Čech closure operator).
In Joshi's Introduction to General Topology, and in most of the literature, this kind of relation is called a nearness relation (page 114). I think one of the first papers on these things was "Nearness--A Better Approach to Continuity and Limits," by P. Cameron, J. G. Hocking, and S. A. Naimpally, which talks about nearness relations on metric spaces.
The definition of continuity in terms of open sets really puzzled me at first. I think the definition in terms of the nearness relation is much clearer!

Let $X$ and $Y$ be topological spaces. A continuous map from $X$ to $Y$ is a map $f$ with the property that if $x$ touches $A$, then $f(x)$ touches $f(A)$.

The definitions of convergence and Hausdorffness are quite pretty, in my opinion, and the definition of connectedness is very intuitive. (WARNING: I'm kinda rusty at this, so these definitions may not be correct.)

The sequence $(x_n)$ converges to the point $x$ if $x$ touches every subsequence of $(x_n)$.
The topological space $X$ is Hausdorff if for any two distinct points $w, x \in X$, there is a subset $A$ of $X$ such that $w$ doesn't touch $A$ and $x$ doesn't touch the complement of $A$.
The topological space $X$ is disconnected if it has two subsets $A$ and $B$, with $A \cup B = X$, such that no point of $A$ touches $B$ and no point of $B$ touches $A$.

A: Without specifying a precise answer to the question, I am surprised that there has been so little emphasis on continuity as the motivating concept for topology - topological spaces seem to me to have been designed, so to speak, to capture the notion of continuity in as much generality as seemed possible at the time, and particularly in non-metric contexts - and incidentally clarifying some proofs by throwing away the metric structure.
What can we recover from the epsilon-delta formulation of continuity if we don't allow measurement? It is possible that this question is more readily answered by reference to closed sets than to open ones.
Clearly the concept then takes off in all sorts of directions, where intuitions motivated by metrics are confounded (as mine was initially with the Zariski Topology).
A: Two platitudes: 
(1) On a metric space, $\mathbb{R}$-valued functions which are continuous in the ($\epsilon$-$\delta$)-sense are the same as those for which the preimage of an open is open. So one can achieve the aim of discussing continuity by using open sets. 
(2) The standard open sets in a metric space satisfy the axioms for a topology.
However, the open sets in a metric space satisfy many other properties too (Hausdorff, etc.).
So - as a former colleague of mine pointed out - to motivate our definition we ought to say why we can't reasonably drop one of the axioms for a topology - the intersection axiom, say. After all, our examples will still satisfy the axioms, and we'll still be able to prove some standard lemmas about spaces and continuous functions. 
The answer, I think, is that continuity really ought to be local: a function is continuous if it's continuous when restricted to each of the sets making up an open cover. In proving this, we use both the union and intersection axioms.
A: Topology can be defined directly, without open sets, as the study of "metric spaces without the metric", i.e., modulo metric deformations or homeomorphism.  This matches reasonably well the intuition of a qualitative geometry, insensitive to stretching and bending.
One can then prove that the structure of open sets is a complete invariant (since we start from metrizable spaces), and one can observe, with some experience, that reasoning about the topology of metric spaces (i.e., proofs of properties that are invariant under deformation or homeomorphism) can be formulated directly in terms of this invariant.  In other words, not only the topologically invariant maps but the constructions of those maps descend to the category of topological spaces in terms of open sets.  This means that we can work natively in manifestly topologically invariant terms provided that the invariant thing --- the structure of open sets --- is taken as the object of study.  This is a rare case of a total or near-total success of the Erlangen program, where thinking in terms of that which is invariant really suffices for the original purposes of the subject.
(I say near-total, but don't know of any example where topologically invariant properties of a metric spaces are most easily proved using one or more metrics.)
Once topology is set up in terms of open sets one can look at examples beyond the motivating intuition, such as Zariski topology, the long line or pathological spaces.  As far as those extensions start to challenge the adequacy of the open-set formalism it is because they are based on phenomena different from the stretching and bending ideas abstracted from picturesque low-dimensional situations. 
A: To me, like someone said before, it's because of the way you define continuity in a metric space... We've all heard (i think) that topology is a game "where bending and stretching is allowed but tearing is not", this is precisely what a continuous function does, so now, given a weird rubberish material (our topological space) how do we define continuity? Thinking back to metric spaces we find our $\epsilon$'s and $\delta$'s which are really just instructions on how to construct our space, they give us a definition of "nearness" (or "separation" some might say), so we construct our pieces of space with that definition in mind, we want a notion of when points are "near" (or "far") and we call those sets of relativily-near points "open"... 
To me it also helps to think of manifolds, like the usual example, the earth! it looks flat, but that's because we're looking at it locally, and what does "local" mean? an open set! 
A: I found that the comment box underneath andrews response wasnt large enough for what i had to say. I think that before i continue in my answer i should mention that i study homotopy theory, and maybe that is why i dont really care about motivating the "original" definition of a topological space. In homotopy theory, and perhaps any geometric flavor of topology, we work with things that have the homotopy type of a CW-complex, these may be much easier to motivate.
I think that the best way to motivate the definition in terms of open sets is historical (i think this is often the case, when you look at what people were thinking about or the problems they were trying to solve or overcome the definition might become clearer). When people started writing down the definition of what a topological space was there was a strong penchant for axioms and set theory. This is the flavor of the definition in terms of open sets. The definition that we have in terms of open sets was gotten after a bit of hard work with bad definitions. There was a lot of change in the culture of mathematics at the turn of the century and a lot of things had to be reworked and made rigorous.  Perhaps i have the facts wrong, but it makes some sense this way even if i am mistaken.
One of my instructors frequently answers questions by saying things like we dont care about that or that is a bad question, which i feel is a legitimate response. The point is that there is a lot of mathematics to be done, a lot of really beautiful important mathematics. You can't really do all of it in a lifetime, so it is probably good to accept some simplifying assumptions like your ring is Noetherian or your space has the homotopy type of a CW-complex. The objects you are ignoring are not that natural to begin with and the things you are looking at are really much more important. In the end the questions we don't answer about the topologists sine curve won't really matter (...I think?) How could you hope to answer a question about some pathological special example with a tool that is meant to capture intuition?
since i dont know how to save this answer as a draft i will just have to settle for coming back to edit it later
