First of all, I was somewhat confused about terminologly when I originally wrote the question. Quasivarieties are algebraic (in the sense of Adámek-Herrlich-Strecker), so $\mathrm{Top}^{\mathrm{op}}$ is algebraic over $\mathrm{Set}$. This only leaves the strongest form of the question, whether $A^{\mathrm{op}}$ can be monadic over $\mathrm{Set}$ for a topological category $A$ (in other words whether $A$ can be a covariety).
It turns out that this can happen; there is a non-trivial topological covariety. Since topological categories are always complete and cocomplete, the result of Vitale mentioned in Ivan's answer implies that a topological category $A$ is a coquasivariety if and only if it satisfies the following:
- there is a regular injective object $I$ that is a regular cogenerator and
- (coregularity) pushouts of regular monomorphisms are regular monomorphisms.
And $A$ is a covariety if it additionally satisfies the following:
- (Barr-coexactness) cocongruences are cokernel pairs.
In the context of topological categories, 1 implies 2 (which seems like something which would be a known fact, but I couldn't find it anywhere): Fix $G : A \to \mathrm{Set}$, a topological functor, and $I$, a regular injective regular cogenerator of $A$.
Lemma. For any objects $B$ and $C$ of $A$, a function $f : GB \to GC$ lifts to a morphism in $A$ if and only if for every $g : C \to I$, $(Gg) \circ f$ lifts to a morphism in $A$.
Proof. Obviously if $f = Gh$ for some morphism $h : B \to C$, then for every $g : C \to I$, $(Gg) \circ f = G(g \circ h)$. Conversely, assume the lifting condition in the statement of the lemma. Let $X = \prod_{g : C \to I} I$. Let $i : C \to I$ be the canonical map satisfying $\pi_g \circ i = g$ for each projection $\pi_g : \prod_{g : C \to I} I \to I$. Note that since $I$ is a regular cogenerator, $i$ is a regular monomorphism (i.e., an isomorphism onto its image). By our assumption and the universal property of products, we get a morphism $h : B \to X$ satisfying that for each projection $\pi_g : X \to I$, $G(\pi_G \circ h) = (Gg) \circ f$. The set-theoretic image of $h$ is contained in the set-theoretic image of $i$. Therefore we have a morphism $i^{-1} \circ h : B \to C$, but this is precisely the required lift of $f$. $\square$
The lemma implies that the isomorphism type of any object $B$ in $A$ is uniquely determined by its underlying set $GB$ and the set of 'continuous' maps from $B$ to $I$. Moreover, we get that a map $f : B \to C$ is a regular monomorphism if and only if every $g : B \to I$ extends to an $h : C \to I$ (i.e., an $h$ such that $g = h \circ f$).
Proposition. $A$ is coregular (and so in particular $A^{\mathrm{op}}$ is a coquasivariety).
Proof. Since $G$ is topological, $A$ is complete and cocomplete, so we only need to verify that pushouts of regular monomorphisms are regular monomorphisms. Let $f : B \to C$ be a regular monomorphism, let $g : B \to D$ be an arbitrary morphism, and let $E$ be the pushout in the following diagram:
$\require{AMScd}$
\begin{CD}
B @>g>> D\\
@V f V V @VV i_D V\\
C @>>i_C> E
\end{CD}
We need to show that $i_D$ is a regular monomorphism. $G$ is a continuous functor and preserves monomorphisms, so we have that the underlying set of $E$ is the pushout of $Gf : GB \to GC$ and $Gg : GB \to GD$. In other words we can identify it with $D \sqcup (C \setminus f[B])$ (abusing notation a little bit). By the lemma, all we need to show is that every 'continuous' map from $D$ to $I$ extends to a 'continuous' map from $E$ to $I$ (by the comment before the statement of the proposition). Fix $h : D \to I$. Since $I$ is a regular injective object, we can find $i : B \to I$ such that $i \circ f = h \circ g$. Let $j: E \to I$ be the map given by the universal property of $E$ applied to $h$ and $i$. This is the required extension of $h$. Since we can do this for every $h : D \to I$, $g : D \to I$ is a regular monomorphism. $\square$.
So this means that all we need to do is find a topological category with a regular injective regular cogenerator in which cocongruences are cokernel pairs. I originally worked out the following example as a subcategory of the category of pseudotopological spaces, but then I realized it was just equivalent to the category of pretopological spaces. Recall that a pretopological space is a set $X$ together with a function $N$ from $X$ to filters on $X$ satisfying that for each $a \in X$, $X \in N(a)$ and $a \in B$ for every $B \in N(a)$. The elements of $N(a)$ are called neighborhoods of $a$. A function $f : A \to B$ is continuous if for each $a \in A$, the preimage of any neighborhood of $f(a)$ is a neighborhood of $a$. Let $\mathrm{PrTop}$ denote the category of pretopological spaces with continuous maps and let $G : \mathrm{PrTop} \to \mathrm{Set}$ be the obvious forgetful functor.
Given pretopological spaces $A$ and $B$, a map $f : A \to B$ is a regular monomorphism if and only if it is an injection and for each $a \in A$, $N(a) = \{f^{-1}[U] : U \in N(f(a))\}$ (which is a filter since $f$ is an injection). Given a pretopological space $B$ and a subset $X$ of $GB$, this also describes the canoncial subspace pretopology on $X$ induced by the pretopology on $B$.
Let $I$ be the pretopological space with underlying set $\{0,1,2\}$ with $N(0)$ the filter generated by $\{0,1\}$ and $N(1) = N(2) = \{\{0,1,2\}\}$.
Proposition. $I$ is a regular injective object in $\mathrm{PrTop}$.
Proof. Let $B$ be a pretopological space and let $A$ be a subspace of $B$. Let $f : A \to I$ be a continuous map. Let $g : B \to I$ be defined by $g(x) = f(x)$ if $x \in A$ and $g(x) = 1$ if $x \notin A$. We need to verify that $g$ is continuous. Fix $b \in B$. If $b \notin A$ or if $b \in A$ but $f(b) \neq 0$, then $N(g(b)) = \{I\}$, so the preimage of any neighborhood of $g(b)$ is a neighborhood of $b$. If $b \in A$ and $f(b) = 0$, then we have that $\{0,1\}$ is the only non-trivial neighborhood of $f(b)$. Since $f$ is continuous, $f^{-1}[\{0,1\}]$ is a neighborhood of $b$ in $A$. Since $A$ is a subspace of $B$, there is some neighborhood $U$ of $b$ in $B$ such that $A \cap U = f^{-1}[\{0,1\}]$, but $g^{-1}[\{0,1\}] = f^{-1}[\{0,1\}] \cup (B \setminus A)$, so we have that the $g$-preimage of the only non-trivial neighborhood of $g(b) = f(b)$ is a neighborhood of $b$ in $B$. Since we can do this for any such point, $g$ is continuous. $\square$
Proposition. $I$ is a regular cogenerator of $\mathrm{PrTop}$.
Proof. Fix a pretopological space $A$. Let $X = \prod_{f : A \to I} I$ and let $g : A \to X$ be the canonical map satisfying $\pi_f \circ g = f$ for each $f : A \to I$.
First note that for any set $B \subseteq A$, the indicator function of $B$ (thought of as a $\{0,1\}$-valued function to $I$) is continuous, so continuous maps from $A$ to $I$ separate points and $g$ is an injection. Finally, it is immediate that for any $a \in A$ and $U \in N(a)$, the map $f : A$ defined by $f(a) = 0$, $f(x) = 1$ for $x \in U \setminus \{a\}$, and $f(x) = 2$ otherwise is continuous. Therefore for any $a \in A$ and $U \in N(A)$, we have that $g[U]$ is a neighborhood of $g(a)$ in the subspace $g[A]$ and so $g$ is an isomorphism onto its image, which is a regular subobject of a power of $I$. Since we can do this for any $A$, $I$ is a regular cogenerator of $\mathrm{PrTop}$. $\square$
Now we just need to show that cocongruences are cokernel pairs. A cocongruence on an object $A$ is a map $r : A \oplus A \to R$ satisfying that for any object $B$, the set
$$
\{(f,g) \in \mathrm{hom}(A,B)^2 : (\exists h : R \to B)f \oplus g = h \circ r\}
$$
is an equivalence relation on $\mathrm{hom}(A,B)$. This is also witnessed by maps $e : R \to A$, $s : R \to R$, and $t : R \to R \oplus_A R$ corresponding to reflexivity, symmetry, and transitivity of the equivalence relation). Morally speaking, a cocongruence on $A$ is something that looks like it should be the pushout $A \oplus_B A$ for some regular subobject $B$ of $A$. What it means for this to be a cokernel pair is that it actually is. $R$ is a cokernel pair if when we let $i:E \to A$ be the equalizer of the component maps $r_0,r_1 : A \to R$, then $R$ is the pushout in the following diagram:
$\require{AMScd}$
\begin{CD}
E @>i>> A\\
@V i V V @VV r_1 V\\
A @>>r_0> R
\end{CD}
This of course fails in $\mathrm{Top}$ and the key is that pushouts in $\mathrm{PrTop}$ are 'floppier' than pushouts in $\mathrm{Top}$, so the only way to have a cocongruence on $A$ is to actually specify a subspace $B$ of $A$ and let $R = A \oplus_B A$.
It will be useful in a minute to have a precise characterization of pushout pretopologies over subspaces. Fix a pretopological space $A$ and a subspace $B \subseteq A$. The pretopology on $A \oplus_B A$ is the finest (i.e., biggest neighborhood filters) pretopology that makes the two inclusion maps $i_0,i_1 : A \to A \oplus_B A$ continuous. This is accomplished by letting neighborhoods of $a \in i_0[B]=i_1[B]$ be sets $U \subseteq A \oplus_B A$ satisfying that $i_0^{-1}[U]$ and $i_1^{-1}[U]$ are neighborhoods of $i_0^{-1}(a) = i_1^{-1}(a) \in B$ and neighborhoods of $a \in i_0[A \setminus B]$ be sets $U \subseteq A \oplus_B A$ satisfying that $i_0^{-1}[U]$ is a neighborhood of $i_0^{-1}(a)$.
(Note that you can already see the floppiness of pretopological pushouts relative to topological pushouts in the example of gluing two two-point indiscrete spaces at a point. Topologically this gives you the three-point indiscrete space, but pretopologically this gives you $\{0,1,2\}$ where $N(0)$ is generated by $\{0,1\}$, $N(1)$ is generated by $\{0,1,2\}$, and $N(2)$ is generated by $\{1,2\}$. As wrote up in an MSE answer recently, this is related to the easiest example of a cocongruence in $\mathrm{Top}$ that is not a cokernel pair..)
One thing to note for the following proof is that any topological functor $G : A \to \mathrm{Set}$ maps cocongruences to cocongruences. Cocongruences in $\mathrm{Set}$ are precisely pushouts of subsets.
Proposition. Cocongruences in $\mathrm{PrTop}$ are cokernel pairs.
Proof. Let $r : A \oplus A \to R$ be a cocongruence with component maps $r_0,r_1 : A \to R$. Since $R$ is a cocongruence, $r_0$ and $r_1$ are both regular monomorphisms (i.e., isomorphisms onto their image subspaces). By the comment just before the proposition, $Gr : G(A \oplus A) \cong GA \sqcup GA \to GR$ is a cocongruence in $\mathrm{Set}$. Let $Gi : GB \to GA$ be the equalizer of $Gr_0$ and $Gr_1$ and let $B$ be the corresponding pretopological subspace (with inclusion map $i : B \to A$). By the universal property of pushouts, we get a map $f : A \oplus_B A \to R$ which is a bijection of underlying sets (and more specifically is an isomorphism of $G(A \oplus_B A)$ and $GR$ as cocongruences in $\mathrm{Set}$). Therefore we know that the pretopology on $R$ is at least as coarse as the pushout pretopology and we need to show that it is no coarser. In particular this means that we need to show that for any point $a$ in $R$ (whose underlying set we can think of as $GA \sqcup_{GB} GA$), any neighborhood of $a$ in the pretopology $A \oplus_B A$ is a neighborhood of $a$ in the pretopology $R$.
Let $C$ be the pushout in the following diagram:
$\require{AMScd}$
\begin{CD}
A @>r_0>> R\\
@V r_1 V V @VV j_0 V\\
R @>> j_1 > C
\end{CD}
Since $R$ is a cocongruence, there must be a map $t : R \to C$ satisfying that $j_0 \circ r_1 = t \circ r_1$ and $j_1 \circ r_0 = t \circ r_0$. (This is the map witnessing transitivity of $R$ in general.)
Identify the underlying set of $C$ with $B \sqcup ((A \setminus B) \times \{0,1,2\})$, where $B \sqcup ((A \setminus B) \times \{0,1\})$ is the image of $j_0$ and $B \sqcup ((A \setminus B) \times \{1,2\})$ is the image of $j_1$. Let $C_k = B \sqcup ((A \setminus B) \times \{k\})$ for each $k < 3$. Note that there is only one relevant pretopology on each $C_k$ (the pretopology corresponding to that on $A$), and this is the subspace pretopology of $C_k$ relative to any of the other pretopologies considered in this argument. Note also that $C_0 \cup C_2$ is the image of $t$.
Fix a point $a \in R$ and consider $t(a) \in C_0 \cup C_2$, and let $U$ be a neighborhood of $a$ in the pushout pretopology on $C_0 \cup C_2 = B \sqcup ((A \setminus B) \times \{0,2\})$ (i.e., the pretopology on $A \oplus_B A$). Let $V = U \cup ((A \setminus B)\times \{1\})$. We need to argue that $V$ is a neighborhood of $t(a)$ in the pretopology on $C$. We have that for both $k \in \{0,2\}$, if $t(a) \in C_k$, then $U \cap C_k$ is a neighborhood of $t(a)$ in the pretopology on $C_k$. Therefore, if $t(a) \in C_0$, then $(U \cap C_0) \cup ((A \setminus B) \times \{1\})$ is a neighborhood of $t(a)$ in the pretopology coming from $R$ (since $C_0$ is a subspace), and likewise if $t(a) \in C_2$, then $(U \cap C_2) \cup ((A \setminus B) \times \{1\})$ is a neighborhood of $t(a)$ in the pretopology coming from $R$. Therefore, by our characterization of pushout pretopologies, $V$ is a neighborhood of $t(a)$ in the pushout pretopology on $C = C_0 \cup C_1 \cup C_2$. Since $t$ is continuous, this implies that $t^{-1}[V] = t^{-1}[U]$ is a neighborhood of $a$ in $R$. Since we can do this for any $a \in R$ and any neighborhood of $t(a)$ in the pushout pretopology on $C_0 \cup C_2$, we have that the map $f$ is an isomorphism and so in particular $R$ is the cokernel pair of the inclusion map of $B$ into $A$. $\square$
So now we can conclude that $A^{\mathrm{op}}$ is monadic over $\mathrm{Set}$ by Vitale's charcterization. In particular, the monad sends a set $X$ to the pretopological space $I^X$. It should be possible to give an explicit algebraic description of the dual category, analogously to the grids of Barr and Pedicchio, but I don't know a clean presentation. Also, since $\mathrm{Top}$ is a full subcategory of $\mathrm{PrTop}$, this gives a positive answer for several of the categories in my second question.
I'm curious about the extent to which this can be pushed further. For instance, can a category satisfying these conditions be a quasitopos (which $\mathrm{PrTop}$ is not)? Bigger generalizations of $\mathrm{Top}$ (such as $\mathrm{Conv}$ and $\mathrm{PsTop}$) also seem like they should have the requisite 'floppiness' to have all cocongruences be cokernel pairs (which is why I was trying to find an example among subcategories of $\mathrm{PsTop}$), but the existence of a regular injective regular cogenerator is far less clear. This answer to an older question of mine makes me suspect there cannot be any such thing for $\mathrm{PsTop}$.