Like Scott Carter, I sometimes say something like: "I was never any good at [e.g.] writing essays..."
"I was never any good at math(s)" always makes me feel funny because usually the speaker is angling for some kind of commiseration which I simply cannot provide. So I try to change the subject as quickly as possible.

