Given any Hopf algebra $A$ over a field $k$, one can also define the Hopf dual $A^*$ of as follows: Let $A^∗$ be the subspace of the full linear dual of $A$ consisting of elements that vanish on some two-sided ideal of $A$ of finite codimension. Then $A^∗$ has a natural Hopf algebra structure.
Question: Is the Hopf dual of the Hopf dual of $A$ isomorphic to $A$. It is not obvious for me that it is. If not, then do we know in which cases it is true?
I am going to give three counterexamples to your first question. (The third counterexample is courtesy of @Adrien, who did most of the job.) While none of them leads to a full answer of your second question, at least they strongly restrict the possibilities.
I will denote the Hopf dual of a Hopf algebra (or coalgebra) $A$ by $A^o$.
The Hopf dual of the Hopf dual of a Hopf algebra $A$ is not, in general, isomorphic to $A$. Better yet:
Proposition 1. Let $k$ be any field. There exists a Hopf algebra $H$ such that $H$ is infinite-dimensional but $\left(H^o\right)^o$ is $1$-dimensional.
Proof. I hope the following is true -- I am using results from the literature I have never checked myself.
The paper A. J. Berrick, The acyclic group dichotomy, arXiv:1006.4009v1, Journal of Algebra, Volume 326, Issue 1, 15 January 2011, pp. 47--58 includes a survey of known results about binate groups. In particular, it says (Theorem 2.7 (a)) that every group embeds in a universal binate group. Thus, there exists at least one infinite binate group. Fix such a group, and denote it by $G$. Also, Theorem 2.2 (b) in this paper says that binate groups have no nontrivial finite-dimensional representations over any field. Thus, $G$ has no such representations.
Now, let $H$ be the group ring $k\left[G\right]$ regarded as a Hopf algebra. Let $f \in H^o$. By the definition of $H^o$, this means that $f$ is a $k$-linear map $H \to k$ that vanishes on some two-sided ideal $I$ of $H$ of finite codimension. Consider this $I$. The finite-dimensional quotient space $H/I$ is an $H$-module, thus a representation of $G$, and therefore must be the trivial representation of $G$ (since $G$ has no nontrivial finite-dimensional representations). Thus, each $g \in G$ acts as the identity on $H/I$. In other words, each $g \in G$ and $h \in H$ satisfy $gh \equiv h \mod I$. Applying this to $h = 1$, we conclude that each $g \in G$ satisfies $g \equiv 1 \mod I$. In other words, each $g \in G$ satisfies $g - 1 \in I$. Therefore, the counit $\varepsilon$ of $H$ satisfies $\operatorname{Ker}\varepsilon \subseteq I$ (since the vector space $\operatorname{Ker}\varepsilon$ is spanned by the $g-1$ for $g \in G$). Hence, the map $f$ vanishes on $\operatorname{Ker}\varepsilon$ (since it vanishes on $I$), and therefore factors through the projection map $H \to H/\operatorname{Ker}\varepsilon$. But factoring through this projection map is tantamount to factoring through $\varepsilon : H \to k$ (since $H/\operatorname{Ker}\varepsilon \cong k$). Thus, $f$ factors through $\varepsilon : H \to k$. Therefore, $f$ is a multiple of $\varepsilon$.
Now, forget that we fixed $f$. Thus, we have shown that every $f \in H^o$ is a multiple of $\varepsilon$. Hence, the Hopf dual $H^o$ of $H$ is spanned by $\varepsilon$ (indeed, it is easy to see that $\varepsilon$ indeed lies in $H^o$). Therefore, $H^o$ is $1$-dimensional, and isomorphic to the trivial Hopf algebra $k$. Thus, its dual $\left(H^o\right)^o$ is isomorphic to the Hopf algebra $k^o \cong k$, hence also $1$-dimensional. But the group $G$ is infinite, and thus its group ring $H$ is infinite-dimensional. This proves Proposition 1. $\blacksquare$
The $H$ constructed in this proof is a fairly wild object by the criteria of Hopf algebra theory or even combinatorics. In particular, $G$ is not finitely generated (again, see the above-cited paper), whence the algebra $H$ is not finitely generated either. A first step in improving the above proposition would be to see if requiring $H$ to be finitely generated helps. Finitely generated groups can still be fairly perverse -- e.g., the Higman group has no faithful finite-dimensional representation, so one would expect $H^o$ to "forget" some part of $H$, but this is no longer completely automatic.
There is another way of proving Proposition 1 in the case when $k = \mathbb{C}$. I will actually show the following stronger fact in this case:
Proposition 2. Let $k$ be a subfield of $\mathbb{C}$. Then, there exists a Hopf algebra $H$ such that $H$ is infinite-dimensional but $\left(H^o\right)^o$ is $1$-dimensional, and furthermore, $H$ is finitely generated as an algebra.
Proof of Proposition 2. Let $G$ be the Higman group; this is the group with four generators $a,b,c,d$ and four relations \begin{equation} ab = ba^2, \quad bc = cb^2, \quad cd = dc^2, \quad da = ad^2 . \end{equation} This is the same group as what the Wikipedia article calls "Higman group", except that our generators $a,b,c,d$ correspond to $a,d,c,b$ in that article. Theorem 1 in Terry Tao's post Finite subsets of groups with no finite models shows that this group $G$ is infinite. But Remark 2 in the same post shows that this group $G$ has no non-trivial finite-dimensional representations. (Note that the proof Tao gives relies on asymptotics of powers of the matrices representing $a, b, c, d$; this is why I required $k$ to be a subfield of $\mathbb{C}$. But I wouldn't be surprised if the argument can be tweaked to work over any field of characteristic $0$.) From this point on, we can use the very same argument that we made in the proof of Proposition 1 to show that $H$ is infinite-dimensional but $\left(H^o\right)^o$ is finite-dimensional. Finally, the group $G$ is generated by four generators $a, b, c, d$; thus, its group algebra $H = k\left[G\right]$ is generated by eight generators $a, b, c, d, a^{-1}, b^{-1}, c^{-1}, d^{-1}$. Hence, $H$ is finitely generated. This proves Proposition 2. $\blacksquare$
The final counterexample is mostly due to @Adrien. It is, in a sense, the most striking since it shows that $\left(H^o\right)^o \not\cong H$ can happen even if $H$ is a univariate polynomial ring -- more or less the simplest case that isn't finite-dimensional!
Proposition 3. Let $k = \mathbb{C}$. Let $H$ be the polynomial ring $k\left[x\right]$ with its usual Hopf algebra structure (in which $x$ is primitive). Then, $H$ has countable dimension (as $k$-vector space) whereas $\left(H^o\right)^o$ has uncountable dimension.
Proof of Proposition 3. Let $G$ be the additive group $\left(\mathbb{C}, +\right)$ written multiplicatively. For each $\lambda \in \mathbb{C}$, let $\left[\lambda\right]$ be the corresponding element of $G$; thus, $\left[0\right]$ is the identity element of $G$, and $\left[\alpha+\beta\right] = \left[\alpha\right]\left[\beta\right]$ holds for any $\alpha, \beta \in \mathbb{C}$.
For each nonnegative integer $n$, let $f^{\left(n\right)}$ denote the $k$-linear map $H \to k$ sending each monomial $x^m$ to $\delta_{n, m}$ (Kronecker delta). It is well-known that $f^{\left(n\right)} \in H^o$ (since $f^{\left(n\right)}$ annihilates the finite-codimensional ideal $\left(x^{n+1}\right)$ of $H$). Actually, $\left(f^{\left(n\right)}\right)_{n \geq 0}$ is a basis of the graded dual of the graded Hopf algebra $H^{\operatorname{gr} *}$ of $H$. Thus, this family $\left(f^{\left(n\right)}\right)_{n \geq 0}$ spans a Hopf subalgebra of $H^o$, which we denote by $H_P$. Note that this Hopf subalgebra is itself isomorphic to $k\left[x\right]$ (using the isomorphism that sends each $f^{\left(n\right)}$ to $x^n / n!$).
The Hopf dual $H^o$, however, is larger than this. Namely, Example 9.1.7 in Susan Montgomery, Hopf Algebras and Their Actions on Rings shows that $H^o \cong H_P \otimes k\left[G\right]$. More precisely, for each $\left[\lambda\right] \in G$, we can define a map $\phi_{\lambda} : H \to k$ which sends each polynomial $p \in H$ to $p\left(\lambda\right)$. This $\phi_{\lambda}$ is a $k$-algebra homomorphism (it is just the evaluation homomorphism at $\lambda$), and thus is a grouplike element of $H^o$. Moreover, each grouplike element of $H^o$ has the form $\phi_\lambda$ for some $\lambda \in G$ (because a grouplike element of $H^o$ is the same as a $k$-algebra homomorphism $H \to k$, but all $k$-algebra homomorphisms from the polynomial ring $H = k\left[x\right]$ are evaluation homomorphisms). The $k$-linear map \begin{equation} \phi : k\left[G\right] \to H^o, \qquad \left[\lambda\right] \mapsto \phi_{\lambda} \end{equation} is a $k$-algebra homomorphism (this boils down to the identity $\phi_\alpha * \phi_\beta = \phi_{\alpha+\beta}$, which in turn boils down to the binomial formula). Moreover, the elements $\phi_\lambda$ for $\lambda \in G$ are distinct grouplike elements of $H^o$, and thus are linearly independent (due to the known fact that any set of distinct grouplike elements of a coalgebra is linearly independent). Hence, the map $\phi$ is injective. Now, the $k$-algebra homomorphism \begin{equation} A : H_P \otimes k\left[G\right] \to H^o, \qquad f \otimes g \mapsto f \phi\left(g\right) \end{equation} turns out to be an isomorphism of Hopf algebras. This is the full version of the $H^o \cong H_P \otimes k\left[G\right]$ statement I mentioned above. (Montgomery writes it as $H^o \cong H_P \otimes \mathbf{k} G\left(H^o\right)$; here, $G\left(H^o\right)$ denotes the span of the grouplike elements of $H^o$, which is isomorphic to our $k\left[G\right]$ because the grouplike elements of $H^o$ are all of the form $\phi_\lambda$.)
Let $\varepsilon$ denote the counit of the Hopf algebra $H_P$; it sends each $f^{\left(n\right)}$ to $\delta_{n,0}$. Recall again that $k = \mathbb{C}$, so we can use transcendental tools like the exponential map. Now, for each $\lambda \in \mathbb{C}$, we define a $k$-linear map \begin{equation} \psi_\lambda : H_P \otimes k\left[G\right] \to k, \qquad \alpha \otimes \left[\beta\right] \mapsto \varepsilon\left(\alpha\right) \exp\left(\lambda \beta\right) . \end{equation} It is easy to see that this $\psi_\lambda$ is well-defined and a $k$-algebra homomorphism. In view of the Hopf algebra isomorphism $H^o \cong H_P \otimes k\left[G\right]$, we can thus consider $\psi_\lambda$ as a $k$-algebra homomorphism $H^o \to k$. In other words, we consider $\psi_\lambda$ as a grouplike element of the Hopf algebra $\left(H^o\right)^o$. These grouplike elements $\psi_\lambda$ for varying $\lambda \in \mathbb{C}$ are all distinct (indeed, $\psi_\lambda$ sends $\phi_\mu \in H^o$ to $\exp\left(\lambda \mu\right)$, and if you know the values of $\exp\left(\lambda \mu\right)$ for all $\mu \in \mathbb{C}$, then you can recover $\lambda$), and thus are linearly independent (due to the known fact that any set of distinct grouplike elements of a coalgebra must be linearly independent). Thus, we have found uncountably many linearly independent elements of $\left(H^o\right)^o$ (since there are uncountably many $\lambda \in \mathbb{C}$). Therefore, the vector space $\left(H^o\right)^o$ has uncountable dimension. But the vector space $H = k\left[x\right]$ has countable dimension. This proves Proposition 3. $\blacksquare$
I don't know a concrete answer to the second question, but I think this will rarely happen that those are the same (and in fact on the top of my head I can't think of an example where they are apart from the finite dimensional case).
So not a real answer but this is too long for a comment: it's useful to remember that a great deal of the asymmetry between algebras and coalgebras boils down to the fact that a coalgebra is characterized by its finite dimensional comodules, while an algebra is not characterized by its finite dimensional modules. This is relevant to this discussion because $A^\circ$ can also be characterized as the (unique up to iso) Hopf algebra whose category of f.d. comodules is equivalent as a monoidal category with fiber functor to f.d. modules over $A$. This somewhat explains why non-isomorphic Hopf algebra can have isomorphic Hopf duals.
Regarding your first question: the answer is generally no, the restricted dual of the restricted dual of $A$ is generally not isomorphic to $A$: $$ (A^{\circ})^\circ\ncong A $$ as has already been indicated by the counterexamples of darij grinberg's answer.
More counterexamples can be constructed when the restricted dual (or: Sweedler's dual) $A^\circ$ is trivial while $A$ is not:
Consider the case when the algebra $A$ has no finite dimensional representations, except the zero vector space $V=\{0\}$. Now, recall that
$f\in A^\circ$ $\Longleftrightarrow$ $\dim(A\rightharpoonup f)<\infty$
where the action of the $A\rightharpoonup f$ module is defined by $(a\cdot f)(b)=f(ba)$ for all $a,b\in A$ and $f\in A^\circ$ (this is a standard result, see for example Montgomery's book, Lemma 9.1.1, p. 149). Thus,
If $A$ has no finite dimensional representations apart from the zero vector space $V=\{0\}$, then $A^\circ=\{0\}$.
Regarding your second question, unfortunately i have no examples or conditions available (and i think it is a difficult task to find something similar in the literature).
