To elaborate on Plyush Grover's comment, large swaths of theoretical physics can be considered "non-rigorous yet inspiring." The adjective "quirky" might not be so apposite, though. But perhaps somewhat in the spirit of the question is Richard Feynman's semi-popular book QED: The Strange Theory of Light and Matter. There are still some open questions about the mathematical consistency of QED.

To elaborate on Piyush Grover's comment, large swaths of theoretical physics can be considered "non-rigorous yet inspiring." The adjective "quirky" might not be so apposite, though. But perhaps somewhat in the spirit of the question is Richard Feynman's semi-popular book QED: The Strange Theory of Light and Matter. There are still some open questions about the mathematical consistency of QED.