There is a refference in my comment above (Lemma 2.1.5 in arxiv.org/pdf/1811.02057.pdf). Let me repeat the argument from there in the answer here. 

Let $X$ be a $\pi$-finite space and $q:X\to pt$ be the projection to the point. We assume that the norm map $Nm_q : q_! \to q_*$ is an isomorphism when evaluated at "trivial modules", which are just functors of the form $q^*X$. We want to prove that it is an isomorphism at every object. 

For an object $X$, the zigzag indetity gives us a retract diagram $q_*X \to q_*q^*q_*X \to q_*X$. Now, the norm map $Nm: q_! q^*q_*X \to q_*q^*q_*X$ is an isomorphism by the assumption, and the naturality of the norm map gives us that the composition 
$q_*X\to q_*q^*q_*X \stackrel{Nm^{-1}}{\to} q_!q^*q_*X \to q_!X \stackrel{Nm}{\to}q_*X$ where the unlabled arrows are the usual units and counits. The naturality of the Norm and the Zigzag identity then imply that this is the identity. So $Nm: q_! X \to q_*X$ is invertible from the left. A similar dual argument with  the Zigzag identity for $q_!$ and $q^*$ gives us that it is right invertible, hence an isomorphism. 

The intuition here is that taking homotopy orbits/fixedpoints twice just enlarge the object so this way we reduce from general module to a trivial one.