In pg. 24 of his book on Galois cohomology, Serre gives the following exercise:

"Give an example of an extension $1 \to P \to E \to G \to 1$ of profinite groups with the following properties:

(i) $P$ is a pro p-group

(ii) $G$ is finite.

(iii) A Sylow p-subgroup of $G$ lifts to $E.$

(iv) $G$ does not lift to $E.$"

I am not sure how this can happen, allow me to explain. Let us start by assuming that all groups involved are finite. Suppose that $S$ is a Sylow $p$-subgroup of $G$ and that it lifts to $E.$ Taking the fiber product along the inclusion $S \to G,$ we get the exact sequence
$$1 \rightarrow P \rightarrow E \times_G S \to S \rightarrow 1.$$ By the assumption that $S$ lifts to $E,$ this exact sequence is split.

If $P$ is an abelian $p$-group, this should imply that there is a lift of $G$ to $E,$ since the map $res:H^2(G,P) \to H^2(S,P)$ is injective by the assumption that $S$ is a Sylow $p$-subgroup.

Now, it seems to me that using the above argument for when $P$ is an abelian $p$-group, we should be able to do some devissage argument to show that given any extension $1 \to P \to E \to G \to 1$ of profinite groups and a lift of a Sylow $p$-subgroup of $G$ to $E$ as in the exercise, there is a lift of $G$ to $E$ as well. But this is in contradiction to the exercise.

Obviously, this means I have made a mistake in my thinking, but I have spent some time on it but can not seem to find it. Could someone help me finding where this mistake is?

**Update**
I now see where my mistake lies. I will post this as an answer tomorrow.