This is a beautiful aspect of the analogy. In fact, viewed as a consequence of previously-established aspects of the analogy, the relationship between linking numbers and abelian integrals is rather simple. As with many ideas in arithmetic topology, the ideas is to replace something analytic (in this case, Gauss's linking integral) with something sufficiently algebraic that we can port it over to the number field situation. The short answer is that both linking numbers and quadratic residue symbols can be computed as cup products in appropriate cohomology groups (here, "appropriate" might be taken to mean that they have already been established as analogous by previous aspects of the analogy). The symmetry of the linking number thus becomes completely analogous to the statement of quadratic reciprocity for primes congruent to 1 mod 4 (for which the analogy is most precise/applicable for ramification reasons).

Here are some rough details:

We begin by observing that in the framework of algebraic topology, the mod-2 linking number of two knots $K$ and $L$ can be computed as a cup product in the relevant cohomology groups: We have $$ \text{lk}(K,L)=[K]\cup [\Sigma_L]\in H_c^3(S^3-L,\mathbb{Z}/2)\cong \{\pm 1\}, $$ where $[K]\in H_c^2(S^3-L,\mathbb{Z}/2)$ and $\Sigma_L$ is the Siefert surface of $L$, so $[\Sigma_L]\in H^1(S^3-L,\mathbb{Z}/2)$.

With an algebraically-analogizable object in hand, we turn to the number field situation. Here, for primes $p$ and $q$, playing the role of the knots, we have a well-established body of arithmetic-topology analogies. Here, instead of $S^3-K$ we have $\text{Spec}(\mathbb{Z}-\{p\}$, whose (etale$^*$) cohomology with coefficients in $\mathbb{Z}/2$, $H^1(\text{Spec}(\mathbb{Z}-\{p\},\mathbb{Z}/2)\cong \text{Hom}(\text{Gal}(\mathbb{Q}(\sqrt{p})/\mathbb{Q}),\mathbb{Z}/2)$ contains the class $[\Sigma]$ of the "Seifert surface" $\Sigma$ corresponding under that isomorphism to the traditional Kummer character $\chi_p$ of $\text{Gal}(\mathbb{Q}(\sqrt{p})/\mathbb{Q}).$ Now, in what amounts to not much more than the standard analysis of quadratic reciprocity using the language of cohomology, we identify an element of $H^2(\text{Spec}(\mathbb{Z}-\{p\}),\mathbb{Z}/2)$ by taking the class of $[q]\in \mathbb{Q}_p^{\times}/\mathbb{Q}_p^{\times 2}\cong H^1(\mathbb{Q}_p,\mathbb{Z}/2)$, and then using the map $H^1(\mathbb{Q}_p,\mathbb{Z}/2)\to H^2(\text{Spec}(\mathbb{Z}-\{p\}),\mathbb{Z}/2).$ Then, finishing the analogy, we have $$ \left(\frac{q}{p}\right)=[p]\cup [\Sigma_q]\in H^3(\text{Spec}(\mathbb{Z}-\{p\}),\mathbb{Z}/2)\cong \{\pm 1\}. $$ Comparing the two identities for $\text{lk}(K,L)$ and $\left(\frac{q}{p}\right)$ should make the whole thing apparent.

(*): Actually, you need some slight variant of etale cohomlogy which properly compactifies the infinite primes, the details of which I am fuzzy on. All of the ideas here are, to my knowledge, due to Morishita, and best addressed in his article "Analogies between knots and primes, 3-manifolds and number rings," though I'd suggest some other of his papers as prerequisites.

This is a beautiful aspect of the analogy. In fact, I think it's important and cool to view it as a consequence of previously-established aspects of the analogy, rather than a new analogy-by-fiat. Under this light, the relationship between quadratic residues and abelian integrals is rather simple. As with many ideas in arithmetic topology, the ideas is to replace something analytic (in this case, Gauss's linking integral) with something sufficiently algebraic that we can port it over to the number field situation. The short answer is that both linking numbers and quadratic residue symbols can be computed as cup products in appropriate cohomology groups (here, "appropriate" might be taken to mean that they have already been established as analogous by previous aspects of the analogy). The symmetry of the linking number thus becomes completely analogous to the statement of quadratic reciprocity for primes congruent to 1 mod 4 (for which the analogy is most precise/applicable for ramification reasons).

Here are some rough details:

We begin by observing that in the framework of algebraic topology, the mod-2 linking number of two knots $K$ and $L$ can be computed as a cup product in the relevant cohomology groups: We have $$ \text{lk}(K,L)=[K]\cup [\Sigma_L]\in H_c^3(S^3-L,\mathbb{Z}/2)\cong \{\pm 1\}, $$ where $[K]\in H_c^2(S^3-L,\mathbb{Z}/2)$ and $\Sigma_L$ is the Siefert surface of $L$, so $[\Sigma_L]\in H^1(S^3-L,\mathbb{Z}/2)$.

With an algebraically-analogizable object in hand, we turn to the number field situation. Here, for primes $p$ and $q$, playing the role of the knots, we have a well-established body of arithmetic-topology analogies. Here, instead of $S^3-K$ we have $\text{Spec}(\mathbb{Z}-\{p\}$, whose (etale$^*$) cohomology with coefficients in $\mathbb{Z}/2$, $H^1(\text{Spec}(\mathbb{Z}-\{p\},\mathbb{Z}/2)\cong \text{Hom}(\text{Gal}(\mathbb{Q}(\sqrt{p})/\mathbb{Q}),\mathbb{Z}/2)$ contains the class $[\Sigma]$ of the "Seifert surface" $\Sigma$ corresponding under that isomorphism to the traditional Kummer character $\chi_p$ of $\text{Gal}(\mathbb{Q}(\sqrt{p})/\mathbb{Q}).$ Now, in what amounts to not much more than the standard analysis of quadratic reciprocity using the language of cohomology, we identify an element of $H^2(\text{Spec}(\mathbb{Z}-\{p\}),\mathbb{Z}/2)$ by taking the class of $[q]\in \mathbb{Q}_p^{\times}/\mathbb{Q}_p^{\times 2}\cong H^1(\mathbb{Q}_p,\mathbb{Z}/2)$, and then using the map $H^1(\mathbb{Q}_p,\mathbb{Z}/2)\to H^2(\text{Spec}(\mathbb{Z}-\{p\}),\mathbb{Z}/2).$ Then, finishing the analogy, we have $$ \left(\frac{q}{p}\right)=[p]\cup [\Sigma_q]\in H^3(\text{Spec}(\mathbb{Z}-\{p\}),\mathbb{Z}/2)\cong \{\pm 1\}. $$ Comparing the two identities for $\text{lk}(K,L)$ and $\left(\frac{q}{p}\right)$ should make the whole thing apparent.

(*): Actually, you need some slight variant of etale cohomlogy which properly compactifies the infinite primes, the details of which I am fuzzy on. All of the ideas here are, to my knowledge, due to Morishita, and best addressed in his article "Analogies between knots and primes, 3-manifolds and number rings," though I'd suggest some other of his papers as prerequisites.