Emil Jeřábek has already provided a detailed proof to show that the proposed theory $T$ in not bi-interpretable with $\mathsf{PA}$ (even though it is mutually interpretable with $\mathsf{PA}$).
The point of this answer is to completement Jeřábek's nice answer by indicating that the argument of Theorem 5.1 of the paper $\omega$-models of finite set theory (co-authored with Schmerl and Visser, available here) can be slightly fine-tuned to show the failure of bi-interpretability. At the bottom, the argument is close in spirit to Emil's.
More specifically, the fine-tuning needed in the proof of Theorem 5.1 of the aforementioned paper to conclude the failure of bi-interpretability, is to observe that, using Remark 4.10 of the same paper, there are (many) recursive models other than $V_{\omega}$ of $\mathsf{ZF_{fin}}$ that have a definable linear ordering of order-type $\omega$, and thus they satisfy the proposed theory $T$.