As Kevin said, I'm not sure if you can get a formula in any concrete sense, but there is a way to tell if the modular form that gives rise to $\bar\rho$ is the unique form (of a specific level). Say that $\bar\rho$ takes values in $GL_2(k)$ and that $f \in S_k(N,\mathcal{O})$ gives rise to $\bar\rho$ (so we have that $k$ is the residue field of $\mathcal{O}$). Then you can show (see for instance Section 4.1 of Darmon, Diamond, Taylor- Fermat's Last Theorem) that the number of forms $g$ which are congruent to $f$ mod $p$ (i.e. $f$ and $g$ have the same residual representation), is equal to the rank (as an $\mathcal{O}$-algebra) of a certain completed Hecke ring, $\mathbb{T}$. In particular, if this rank is 1, then $f$ is the unique form which gives rise to $\bar\rho$.

Thanks to the general $R=T$ theorems, one can study the rank of $\mathbb{T}$ by studying the deformation theory of the representation $\bar\rho$. From this point of view, the rank is something you can almost get your hands on (and by "almost", I mean, "isn't completely hopeless"). That's because it is a standard fact that $R$ is a quotient of a power series ring over $\mathcal{O}$ in $d$ variables, where $d$ is the dimension of specific subgroup $H\subset H^1(G_S, ad^0\bar\rho)$. This subgroup $H$ is a so-called "Selmer group" since it is the kernel of a global-to-local map on cohomology, defined by specifying local conditions at all primes in $S$.

There are lots of choices of local conditions, and each choice will lead to a different Selmer group. You can read "Deforming Galois representations and the conjectures of Serre and Fontaine-Mazur" by Ravi Ramakrishna for local conditions that deal with "modular" deformations (the same conditions appear in "On icosahedral Artin representations, II" by Richard Taylor). Now, since you have a local-at-$p$ representation in mind, you might have to alter these local conditions a bit (but not too much).

The point though, is that, if you can show that $H=0$, then we have that $R$ is a quotient of of a power series ring over $\mathcal{O}$ in 0 variables. Since completed Hecke rings are known to be finite flat complete intersections, we conclude that $R = \mathbb{T} = \mathcal{O}$ when $H=0$. As mentioned above, the $\mathcal{O}$-rank of $\mathbb{T}$ corresponds to the number of modular forms giving rise to $\bar\rho$. Thus, we get to the punchline:

$f$ is the unique modular form (of weight $k$ and level $N$) giving rise to $\bar\rho$ if and only if $H=0$.

The natural question to ask now is, "When is $H=0$?" This, of course, is the hard part. If you are willing to allow finitely many additional primes into your ramification set, then using the methods of Ramakrishna (Op. cit.) you can force $H=0$. If you want a modular form of optimal Serre conductor, then I don't know if anybody knows how to do this.

Studying whether or not $H =0$ as you vary the set $S$ is the subject of a paper of mine with Ramakrishna (whose contents are basically my thesis), "New Parts of Hecke Rings". You can look at that for ideas on how to determine (in specific settings) if $H=0$.