This is certainly true. If you choose $\epsilon>0$ smaller than half the injectivity radius of $g$ at $p$ (in particular, sufficiently small that the exponential map $\exp_p:T_pU\to U$ is well-defined for all $v\in T_pM$ of $g$-length less than $\epsilon$), then, on $B^g(p,\epsilon)$ the squared $g$-distance from $p$ is a smooth function $\rho:B^g(p,\epsilon)\to\mathbb{R}$ that can be written in the form
$$
\rho = (y^1)^2 + \cdots + (y^n)^2
$$
where $y = (y^i)$ is a smooth coordinate system on $B^g(p,\epsilon)$ that is centered on $p$.

Hence $\rho$ has an isolated proper local minimum at $p$, with a positive definite Hessian at $p$. This implies, by an elementary argument, that the sublevel sets $\rho \le \delta$, for all $\delta\in(0,\epsilon)$ sufficiently close to zero, are convex in $\mathbb{R}^n$.

**Added remark about estimates:** Actually, if one assumes, for simplicity, that $p=0\in\mathbb{R}^n$, then, for $g = g_{ij}(x)\mathrm{d}x^i\mathrm{d}x^j$, one has a Taylor expansion with remainder of the form
$$
\rho = g_{ij}(0)x^ix^j + g_{ijk}(x)x^ix^jx^k
$$
where the functions $g_{ijk}$ and their first and second derivatives can be bounded in $U$ in terms of $g_{ij}(0)$ and uniform bounds on the first, second, and third derivatives (with respect to the coordinates $x^i$) of the $g_{ij}(x)$ in $U$. Once one has this, the local convexity of the function $\rho$ (which will imply that its sublevel sets near $p=0$ are convex in the usual sense) is a matter of checking whether
$$
H = (H_{ij}) = \left(\frac{\partial^2\rho}{\partial x^i\partial x^j}\right)
$$
is positive semidefinite in the region where $\rho(x)\le\epsilon^2$. Again, this can be checked in terms of the explicit bounds that can be worked out from these formulae, so there certainly will be an estimate of how large one can take $\epsilon$ based on the data that the OP wants to use. Probably, one can do better than this by examining the $g_{ijk}(x)$ more carefully. I would expect that one only needs uniform bounds on $g_{ij}$ and its first (and maybe second) derivatives to get such a bound.