Suppose $\kappa$ is weakly inaccessible. Then it is immediate $L_\kappa$ satisfies pairing, separation, extensionality, regularity, infinity, union, and choice. To see that $L_\kappa$ satisfies Replacement, let $h(x)=min\{\alpha|x\subseteq L_\alpha\}$. Let $X\in L_\kappa$, and $F(X)=\{f(x)|x\in X\}$. Let $Z=\{h(x)|x\in F(X)\}$. Such a set exists as $F(X)\subseteq L_\kappa$. Then let $\lambda$ be the order type of $Z$, and $\eta_{\alpha+1}=min\{\beta>\eta_\alpha|\beta\in Z\}$ be continuous at limits, with $\eta_0=min\,Z$. Then if $Z$ is bounded in $\kappa$, it is clear $F(X)\in L_\kappa$. Else $\kappa=\lim_{\alpha\to \lambda}\eta_\alpha$, yet $\lambda<\kappa$ and $\kappa$ is regular. Contradiction.
Now it remains to show that $L_\kappa$ satisfies power set. Let $Y\subseteq X$. Then $L_\kappa\vDash \exists\delta>|Y|(\delta\text{ is a limit}\ \land\ Y\in L_\delta)$. Let $L_\kappa\vDash M\prec L_\delta$, and $|M|=|Y|$. Then take the transitive collapse $\pi(M)$ and so there is some limit ordinal $\gamma$ such that $\pi(Y)=Y$ and $Y\in L_\gamma$. Hence $L_\kappa\vDash ZFC$, and so we cannot prove the consistency of "There exists a weakly inaccessible cardinal" from $ZFC$. This argument does indeed skirt very close to the argument that $V=L\rightarrow GCH$, yet does not quite prove it. The rest follows from every weakly Mahlo cardinal being weakly inaccessible.
Edit: If you want to avoid the $L$ construction all together, then I suggest we construct a new cumulative hierarchy. We want $W_\alpha$ to be sufficiently large to contain pairing, separation, etc. So, we want $L_\alpha\subseteq W_\alpha$. We also want it satisfy the condensation lemma so we need $W_\alpha=\cup_{\beta<\alpha}W_\beta^{W_\alpha}$.
Let $W_{\alpha+1}=\{x\subseteq W_\alpha\mid\text{$\phi(x,p_0...p_n)^{W_\alpha}$ is a definition for $x,p_0...p_n\in W_\alpha$}\}$. If $x=\{y|\phi(x,p_0...p_n)^{W_\alpha}\}$, then $\forall y\in W_\alpha(y\in x\leftrightarrow \phi(x,p_0...p_n)^{W_\alpha})$ is a definition for $x$, and so $L_\alpha\subseteq W_\alpha$. Conversely, as $W_\alpha$ is transitive $\phi(x,p_0...p_n)^{{W_\beta}^{W_\alpha}}\leftrightarrow \phi(x,p_0...p_n)^{W_\beta}$ whenever $W_\beta\in W_\alpha$, and so $W_\alpha=\cup_{\beta<\alpha}W_\beta^{W_\alpha}$.
From here, simply repeat the argument from before.