I'm no expert in this area, but I do have some background in mathematical logic, so I'll give it a shot.

Indeed, a vast amount of work has been done in this area over the last century (e.g. Godel, Tarski). Your first look should be towards model theory; Wilfred Hodges' "A shorter model theory" is a great place to start and should keep you busy for quite some time. You'll learn about relational structures (a very general type of mathematical structure via which all of the objects you mentioned can be realized), the "first-order languages" associated to these structures, and all of the true, "first-order" statements that you can make about them, using the basic underlying relations and functions on your objects as the primitive elements of your language.

You need to be careful however about what you mean by the "set of all true statements you can make about ...". One must always have in mind a well defined language with which you are allowed to make statements and formulas, and the most commonly studied is first-order languages. I'll leave it to Hodges to explain the particulars, but the defining feature of first-order is, you are only allowed to quantify over *elements* of your structure, *not* subsets. For example, the following is a first-order statement in the (typical) language of fields, let's say about the real numbers $\mathbb{R}$:

"(for all elements x and y of $\mathbb{R}$)( x*y = y*x)"

which is obviously asserting that multiplication in $\mathbb{R}$ is commutative. However, the following is *not* first-order:

"(for all subsets $S$ of $\mathbb{R}$)(if $S$ is bounded from above, then it has a least upper bound)"

which is asserting the completeness property of $\mathbb{R}$. It can be shown in fact, no matter how clever you are, you can't express the completeness property of the reals in a first-order way.

Despite its inherent limitations, there are very good reasons for restricting yourself to first-order languages, whenever possible. Firstly, it is a wonderfully expressive language, and there are in fact a good many interesting mathematical statements one can make with it. But secondly, and more importantly, first-order logic satisfies some extremely useful regularity conditions, ones that higher order languages (e.g. those that allow quantification over subsets) do not, and is in some sense the most robust language that satisfies these conditions. Principal among them are the so-called compactness theorem for first-order logic, and Godel's completeness (not incompleteness!) theorem.

But anyway, check out Hodges' book. He's an extremely readable and enlightening author, especially for a logician.