Maybe the following article http://arxiv.org/abs/1306.5973 (Is mathematical history written by the victors?) and references therein will be useful :-).
Fermat's life and work is carefully investigated in the book "The Mathematical Career of Pierre de Fermat, 1601-1665" by Michael Sean Mahoney: http://press.princeton.edu/titles/5449.html See, however, Gerry Myerson's comment below and https://mathematicswithoutapologies.wordpress.com/2015/03/25/andre-weil-vs-history-of-mathematics/
P.S. I have just found two papers by Klaus Barner (in French) which might be useful (in case you are not already aware of them):
https://eudml.org/doc/10129 (Pierre Fermat Sa vie privée et professionnelle)
http://www.zeitschrift-rechtsgeschichte.de/de/article_id/581 (Fermat et l'affaire Delpoy)
P.P.S. Two more papers by Klaus Barner:
http://www.degruyter.com/view/j/dmvm.2001.9.issue-3/dmvm-2001-0070/dmvm-2001-0070.xml (Das Leben Fermats)
http://www.math-in-europe.eu/images/information_pdf/hist_phil_pdf/hist_pdf/barner.pdf (Pierre de Fermat (1601? - 1665) His life beside mathematics)
Russian science fiction writer Alexander Kazantsev wrote a historical novel about Fermat's life: http://www.litmir.me/bd/?b=261604 (Острее шпаги). A German translation of it does exist: https://www.amazon.de/Alexander-Kasanzew-Sch%C3%A4rfer-als-Degen/dp/3355003638 (Schärfer als Degen).
From the more scientific point of view, maybe the following book will be useful: https://www.amazon.co.uk/Public-Life-Toulouse-1463-1789-Cosmopolitan/dp/0801421918 (Public Life in Toulouse, 1463-1789: From Municipal Republic to Cosmopolitan City, by R.~A.~Schneider).