There is not likely to be a good answer to this question, because of a very annoying Theorem due to Heinzer and Roitman:
If $D$ is any UFD, then there is a PID $R$ containing $D$ which has the same unit group as $D$ and such that every prime of $D$ remains prime in $R$.
Since UFD's with finite unit group are common, this will give lots of examples.
Here is a sketch of the proof of Heinzer and Roitman's result: If $D$ is a UFD, and $a$ and $b$ are relatively prime elements of $D$, set $D_{a,b} = D[x,y]/(ax+by-1)$. One can check that $D_{a,b}$ is a UFD and that the ideals $(a)$ and $(b)$ are comaximal in $D_{a,b}$; one can also check that $D^{\times} = D^{\times}_{a,b}$ and that prime elements of $D$ stay prime in $D_{a,b}$.
Using the $D_{a,b}$ construction and a transfinite induction procedure, we embed $D$ into a ring $R$ such that $R$ is a UFD and any two relatively prime elements of $R$ generate comaximal ideals. Such a UFD is necessarily a PID. One also has $R^{\times} = D^{\times}$, and every prime of $D$ stays prime in $R$. See Section 4 of Heinzer, William; Roitman, Moshe, Principal ideal domains and Euclidean domains having 1 as the only unit, Commun. Algebra 29, No. 11, 5197-5208 (2001). ZBL1094.13532. for the details.
As observed in the comments, if such a ring $A$ is finitely generated over $\mathbb{Z}$, it must be either be an order in the ring of integers in a number field, or else it must be the coordinate ring of an affine curve over $\mathbb{F}_q$.
In the first case, by Dirichlet's unit theorem, the number field $K$ must be an imaginary complkex field. Since PID's are integrally closed, $A$ must be the full ring of integers. The imaginary quadratic number fields with class number $1$ are well known, they are $\mathbb{Z}[\sqrt{-1}]$, $\mathbb{Z}[\sqrt{-2}]$ and $\mathbb{Z}[\tfrac{1+\sqrt{-D}}{2}]$ for $D \in \{ 3, 7, 11, 19, 43, 67, 163 \}$.
In the second case, let $A$ be the coordinate ring of an affine curve $U$ over $\mathbb{F}_q$, and let $X$ be the smooth projective curve completing $U$. By Dirichlet's unit theorem for function fields, $U \setminus X$ is a single closed point $x$; let that point $x$ have residue field $q^f$. Then $\mathrm{Pic}(U) = \mathrm{Pic}(X)/[x]$. We have a short exact sequence
$$0 \to \mathrm{Pic}^0(X) \to \mathrm{Pic}(X) \to \mathbb{Z} \to 0$$
and the image of $[x]$ in $\mathbb{Z}$ is $f$ times the generator, so $\mathrm{Pic}(U)$ has a $\mathbb{Z}/f \mathbb{Z}$ quotient.
Our assumption that $A$ is a PID means that $\mathrm{Pic}(U)$ is trivial, so we deduce that $f=1$ and $x$ is an $\mathbb{F}_q$ point. In that case, we have $\mathrm{Pic}(U) = \mathrm{Pic}^0(X)$, so we must additionally have $\mathrm{Pic}^0(X)$ trivial. In other words, if $J$ is the Jacobian of $X$, we must have $J(\mathbb{F}_q)$ trivial.
Let's assume from now on that $X$ has genus $>0$, since genus $0$ gives us the case $\mathbb{F}_q[x]$, which we already know. Note that, with this assumption, requiring that $J(\mathbb{F}_q)$ be trivial imposes in particular that $X(\mathbb{F}_q)$ has at most one point.
Let $\lambda_1$, $\lambda_2$, ..., $\lambda_{2g}$ be the eigenvalues of $q$-power Frobenius on $X$. Then the size of $J(\mathbb{F}_q)$ is $\prod (\lambda_i-1)$. Each $\lambda_i$ is a complex number of norm $\sqrt{q}$, so $\lambda_i$ is a complex number of norm at least $\sqrt{q}-1$. Thus, all examples have $\sqrt{q}-1 \leq 1$, so $q \in \{ 2,3,4 \}$.
Heinzer and Roitman look at the $q=2$ case and give the examples of $y^2+y = x^3+x^2+1$ (genus one) and $y^2+y = x^5+x^3+1$ (genus two). For $q=3$, I noticed $y^2 = x^3-x+1$ (genus one) and for $q=4$ I noticed $y^2+y = x^3+\omega$ (genus one), where $\omega$ is a root of $\omega^2+\omega+1=0$ in $\mathbb{F}_4$. The eigenvalues of Frobenius in the genus $1$ cases are $\sqrt{2} e^{\pm(2 \pi i)/8} =1\pm i$, $\sqrt{3} e^{\pm(2 \pi i)/12} = \tfrac{3}{2} \pm \tfrac{\sqrt{-3}}{2}$ and $2$ (with multiplicity two). The eigenvalues of Frobenius in the genus $2$ case are $\sqrt{2} e^{\pm(2 \pi i)/24}$ and $\sqrt{2} e^{\pm 7(2 \pi i)/24}$, also known as $(1\pm i)\left( \tfrac{1\pm \sqrt{-3}}{2} \right)$.
I attempted to prove that the curves above were the only examples, but my proof had a gap. Fortunately, pregunton discovered a paper of Mercuri and Stirpe which lists all curves with $J(\mathbb{F}_q)$ trivial (both with $X(\mathbb{F}_q)$ singleton, like we want, and with $X(\mathbb{F}_q)$ empty). The curves listed above are cases (i), (ii), (vi) and (vii) on their list; I believe all the other curves on their list have $X(\mathbb{F}_q)$ empty.
Here is what I can salvage from my computation, which I still like:
There are no examples with $q=4$ and genus $>1$. If there were, then we would have $\lambda_1 = \lambda_2 = \cdots = \lambda_{2g}=2$. But then the number of points in $X(\mathbb{F}_4)$ would be $4-\sum \lambda_i + 1 = 5-4g<0$, a contradiction.
Now let $q$ be $2$ or $3$. We observe that $\sqrt{q}$ must have even multiplicity as an eigenvalue of Frobenius. Proof: We have $\#J(\mathbb{F}_q) = \prod (1-\lambda_j)$. Group together the terms with complex conjguate eigenvalues; then every factor except the one coming from $1-\sqrt{q}$ is a positive real, so $(1-\sqrt{q})$ must be raised to an even power as well. Using $\#J(\mathbb{F}_q) = \prod (\lambda_j-1)$, we see that $-\sqrt{q}$ likewise has even multiplicity.
Therefore, we can write the eigenvalues of Frobenius as $\sqrt{q} e^{\pm i \theta_j}$ for $1\leq \theta_j \leq g$, remembering to take both signs in the exponent even when $\theta_j$ is $0$ or $\pi$. Let $c_j = \cos \theta_j$.
The equation $X(\mathbb{F}_q)=1$ translates to
$$q-2 \sqrt{q} \sum \cos \theta_j +1 =1 \ \implies \ \sum c_j = \tfrac{\sqrt{q}}{2} \quad (\ast)$$
The equation $J(\mathbb{F}_q)=1$ translates to
$$\prod (q+1-2 \sqrt{q} c_j) = 1 \quad (\dagger).$$
We note that $(\ast)$ is a convex polytope whose vertices are $(1,1,\ldots,1, -1, -1, \ldots, -1, \tfrac{\sqrt{q}}{2})$ for $g$ odd and
$(1,1,\ldots,1, -1, -1, \ldots, -1, \tfrac{\sqrt{q}}{2}-1)$ for $g$ even.
We will try to show that $\prod (q+1-2 \sqrt{q} c_j) \geq 1$ everywhere on this polytope. Note that $\log \prod (q+1-2 \sqrt{q} c_j)$ is concave, so it is enough to check the inequality at the vertices.
We compute that the value at the vertices is
$$\begin{array}{c|cc}
& g=2k+1 & g=2k+2 \\
\hline
q=2 & 1 & (3-2 \sqrt{2}) (3 - 2 \sqrt{2} (\tfrac{\sqrt{2}}{2}-1)) \approx 0.657 \\
q=3 & 4^k & 4^k (4-2 \sqrt{3}) (4 - 2 \sqrt{3} (\tfrac{\sqrt{3}}{2}-1)) \approx 2.39 \times 4^k \\
\end{array}.$$
We see that the only solution with $q=3$ is $g=1$ with $c_1 = \tfrac{\sqrt{3}}{2}$.
If $q=2$ and $g=2k+1$, then $(\dagger)$ only occurs at the vertices of the polytope, corresponding to the eigenvalue sequence $\sqrt{2}$ (multiplicity $2k$), $-\sqrt{2}$ (multiplicity $2k$) and $1 \pm i$. However, this would lead to a curve with $1-4k$ points over $\mathbb{F}_4$, so this can only happen for $k=0$ (and thus $g=1$.)
However, I am unclear as to how to deal with the case of $q=2$, $g$ even. When $g=2$, the polytope is just a line segment and $(\dagger)$ occurs at two symmetrically placed points of the line segment, corresponding to $(c_1, c_2) = ( \tfrac{1 \pm \sqrt{3}}{\sqrt{2}}, \tfrac{1 \mp \sqrt{3}}{\sqrt{2}})$. However, for larger $k$, $(\dagger)$ occurs on a little $2k$-dimensional manifold in the corners of the polytope, and it isn't clear to me how to rule out more solutions here.
A fun fact is to note that none of these affine curve examples (except genus zero) will be Euclidean. This is due to the following nice exercise: If $A$ is a Euclidean domain and not a field, then there is some prime $\mathfrak{p}$ of $A$ such that $A^{\times} \to (A/\mathfrak{p})^{\times}$ is surjective. (Hint: Take a non-unit of minimal norm.) Since $U$ has no $\mathbb{F}_q$ points in any of these examples, all of the residue fields $A/\mathfrak{p}$ are proper extensions of $\mathbb{F}_q$, whereas $U^{\times} = \mathbb{F}_q^{\times}$.