I suspect that your confusion is because the proof is relatively long and the ideas behind it are not explained. The third proof is the more interesting proof because it gives all the three parts of Sylow's theorem at once. The particular instance in Herstein is due to H. Wielandt.

One abstract idea used in the proof is that of a group action. Herstein never explicitly mentions this idea; but you ought to learn it as it is very important. I leave the definition for you to look up in wikipedia. A group, for example, acts on the set of cosets formed by a subgroup. Another more interesting action is the action of the group on the set of conjugates of any particular subgroup.

The ideas of the three stages of proof can be explained as follows:

1. Write out the counting arguments, and derive Sylow's first theorem from Cauchy's theorem.

2. Consider the action of $G$ on all the $p$-Sylow subgroups and use this action to prove the second theorem of Sylow, that all the Sylow subgroups are conjugate to each other.

3. As a fall-out of the argument in the second step, you will get Sylow's third result.

Unfortunately a proper reference does not come to my mind. But I think you can glue up the proof in a more conceptual way in three steps as above, from the proof of Sylow Theorem -II given in section 1.13 of Jacobson's Basic Algebra -I. That will show you how to use group actions. Once you understand the formalism, turn to exercises 12-17 of the same section, and you can get the proof of Sylow's theorem as in Herstein, but in a more instructive way.

I recall that I had indeed seen H. Wielandt's proof written in a very compact way in some standard algebra book; but for the life of mine I can't remember where. The best approximation that I was able to locate was the above citation of Jacobson's book. Perhaps you could also check out Wielandt's original paper.

If you are in need of applications, Sylow's three theorems are a very interesting tool to classify groups of low cardinality. Some exercises are given in Herstein's book itself. You could start there. The case of groups of order 15 is quite instructive. To deal with non-abelian groups, you would also need the notion of semidirect product, which is given in Jacobson's book I cited above. With this, you can classify all groups up to order 59. This is a very interesting exercise to work out, if you have got the time. The important point here is that all three parts of Sylow's theorem are needed for this job, and only the third proof gives all three parts.

For groups of order 60, check out Michael Artin's algebra book.

As Robin Chapman has given an elegant and compact proof, I content myself with answering your query about applications. Sylow's three theorems are a very interesting tool to classify groups of low cardinality. Some exercises are on this given in Herstein's book itself. Upto groups of order 60, you can use just the three theorems of Sylow and classify them as direct or semi-direct products. Here all three theorems are needed; only the third proof of Herstein proves all three.

The case of groups of order 60 is a bit intricate; the appropriate reference is M. Artin's Algebra book.

I suspect that your confusion is because the proof is relatively long and the ideas behind it are not explained. The third proof is the more interesting proof because it gives all the three parts of Sylow's theorem at once. The particular instance in Herstein is due to H. Wielandt.

One abstract idea used in the proof is that of a group action. Herstein never explicitly mentions this idea; but you ought to learn it as it is very important. I leave the definition for you to look up in wikipedia. A group, for example, acts on the set of cosets formed by a subgroup. Another more interesting action is the action of the group on the set of conjugates of any particular subgroup.

The ideas of the three stages of proof can be explained as follows:

1. Write out the counting arguments, and derive Sylow's first theorem from Cauchy's theorem.

2. Consider the action of $G$ on all the $p$-Sylow subgroups and use this action to prove the second theorem of Sylow, that all the Sylow subgroups are conjugate to each other.

3. As a fall-out of the argument in the second step, you will get Sylow's third result.

Unfortunately a proper reference does not come to my mind. But I think you can glue up the proof in a more conceptual way in three steps as above, from the proof of Sylow Theorem -II given in section 1.13 of Jacobson's Basic Algebra -I. That will show you how to use group actions. Once you understand the formalism, turn to exercises 12-17 of the same section, and you can get the proof of Sylow's theorem as in Herstein, but in a more instructive way.

I recall that I had indeed seen H. Wielandt's proof written in a very compact way in some standard algebra book; but for the life of mine I can't remember where. The best approximation that I was able to locate was the above citation of Jacobson's book. Perhaps you could also check out Wielandt's original paper.

I suspect that your confusion is because the proof is relatively long and the ideas behind it are not explained. The third proof is the more interesting proof because it gives all the three parts of Sylow's theorem at once. The particular instance in Herstein is due to H. Wielandt.

One abstract idea used in the proof is that of a group action. Herstein never explicitly mentions this idea; but you ought to learn it as it is very important. I leave the definition for you to look up in wikipedia. A group, for example, acts on the set of cosets formed by a subgroup. Another more interesting action is the action of the group on the set of conjugates of any particular subgroup.

The ideas of the three stages of proof can be explained as follows:

1. Write out the counting arguments, and derive Sylow's first theorem from Cauchy's theorem.

2. Consider the action of $G$ on all the $p$-Sylow subgroups and use this action to prove the second theorem of Sylow, that all the Sylow subgroups are conjugate to each other.

3. As a fall-out of the argument in the second step, you will get Sylow's third result.

Unfortunately a proper reference does not come to my mind. But I think you can glue up the proof in a more conceptual way in three steps as above, from the proof of Sylow Theorem -II given in section 1.13 of Jacobson's Basic Algebra -I. That will show you how to use group actions. Once you understand the formalism, turn to exercises 12-17 of the same section, and you can get the proof of Sylow's theorem as in Herstein, but in a more instructive way.

I recall that I had indeed seen H. Wielandt's proof written in a very compact way in some standard algebra book; but for the life of mine I can't remember where. The best approximation that I was able to locate was the above citation of Jacobson's book. Perhaps you could also check out Wielandt's original paper.

If you are in need of applications, Sylow's three theorems are a very interesting tool to classify groups of low cardinality. Some exercises are given in Herstein's book itself. You could start there. The case of groups of order 15 is quite instructive. To deal with non-abelian groups, you would also need the notion of semidirect product, which is given in Jacobson's book I cited above. With this, you can classify all groups up to order 59. This is a very interesting exercise to work out, if you have got the time. The important point here is that all three parts of Sylow's theorem are needed for this job, and only the third proof gives all three parts.

For groups of order 60, check out Michael Artin's algebra book.