Let $G$ be a compact Lie group acting on a connected topological manifold $M$ with boundary. Suppose the action on one boundary component is trivial. Does it follow that the action on the whole of $M$ is trivial as well?

If $M$ and the action map $G\times M\to M$ are smooth, it is not too difficult to show that the answer is positive. Indeed, let $X$ be the set of all fixed points $x\in M$ of the action such that the action of $G$ on $T_xM$ is trivial. This set is closed, so it suffices to show it is open and non-empty (since $M$ is connected). To do so take a Riemannian metric on $M$ and average it to get a $G$-invariant metric. Using this one can show that $X$ contains the boundary component on which $G$ acts identically, so $X$ is non-empty. Moreover, if $x\in X$, then any $g\in G$ acts identically on a neighborhood of $x$ since $g(exp(v))=exp(dg_x v)$ for all $v$ in a sufficiently small neighborhood of $0\in T_xM$.

However, this argument uses smoothness and it is not clear if it can be adapted to the topological case.