As stated, I do not believe that the answer has a reasonable answer. Take as an example the theory of groups, which has both an axiomatization with 1 axiom an another with 3 (4 if you count closure). Is the 1-axiom version really `better', even from an information-theoretic point-of-view? You need to actually ask which encoding of these 2 (in some system) is shortest - and for all intents and purposes, the 3-axiom version will be shorter.

For the case where you have an infinite number of axioms, things get more complicated, as you need to make sure you still have a proper information measure (i.e. of total weight $\leq 1$), which is not always easy to achieve in a natural manner.

On the other hand, suitably rephrased, I do believe your general question has a positive answer. Just don't expect there to be a unique minimal system, in the same way one cannot hope to define the 'unique' simplification of an expression (Google for "understanding expression simplification" if you want details).

As stated, I do not believe that the answer has a reasonable answer. Take as an example the theory of groups, which has both an axiomatization with 1 axiom, and another with 3 (4 if you count closure). Is the 1-axiom version really `better', even from an information-theoretic point-of-view? You need to actually ask which encoding of these 2 (in some system) is shortest - and for all intents and purposes, the 3-axiom version will be shorter.

For the case where you have an infinite number of axioms, things get more complicated, as you need to make sure you still have a proper information measure (i.e. of total weight $\leq 1$), which is not always easy to achieve in a natural manner.

On the other hand, suitably rephrased, I do believe your general question has a positive answer. Just don't expect there to be a unique minimal system, in the same way one cannot hope to define the 'unique' simplification of an expression (Google for "understanding expression simplification" if you want details).