A great work on this has been done by Imre Ruzsa; see, for instance, his paper ["Many differences, few sums"][1] in  *Ann. Univ. Sci. Budapest. Eötvös Sect. Math.* **51** (2008), 27–38 (2009).

As a very brief answer to your question, you cannot have a set $A$ of cardinality $N$ with $|2A|\sim N$ and $|A-A|\sim N^2$ since if $|2A|=\alpha N$, then
  $$ \sqrt\alpha N \le |A-A| \le \alpha^2 N; $$
you will find this inequality in the aforementioned paper by Ruzsa.

---

The only way to construct sets with many differences and few sums I can think of (but perhaps, not the only one known to the mankind) is to use the tensor power trick. Start with you favorite set $A_0$ with $|2A_0|=\alpha|A_0|$ and $|A_0-A_0|=\beta|A_0|$, and consider the cartesian power $A:=A_0^k$ with a large $k$. You have $N=|A|=|A_0|^k$, $|2A|=\alpha^k N$, and $|A-A|=\beta^k N$; hence, if $A_0$ is chosen so that $\alpha<\beta$, then $|A-A|$ is much large than $|2A|$ (for $k$ large).

[1]: http://www.ams.org/mathscinet/search/publdoc.html?arg3=&co4=AND&co5=AND&co6=AND&co7=AND&dr=all&pg4=AUCN&pg5=TI&pg6=PC&pg7=ALLF&pg8=ET&review_format=html&s4=Ruzsa%2C%20I%2A&s5=Sums%20and%20differences&s6=&s7=&s8=All&vfpref=html&yearRangeFirst=&yearRangeSecond=&yrop=eq&r=1&mx-pid=2567492