What does the KL being symmetric tell us about the distributions? Suppose two probability density functions, $p$ and $q$, such that $\text{KL}(q||p) = \text{KL}(p||q) \neq 0$. Intuitively, does that tell us anything interesting about the nature of these densities? 
 A: One example where this happens is when $P$ and $Q$ are "antipodal" distributions -- say, $p=(p_1,\ldots,p_n)$, $q=(q_1,\ldots,q_n)$,
$q_i=1-p_i$, and 
$$P=\mathrm{Ber}(p_1)\times\mathrm{Ber}(p_2)\times\ldots
\mathrm{Ber}(p_n)$$
and $Q$ is defined analogously. Then $KL(P||Q)=KL(Q||P)$. These play a role in optimal decision theory, see
http://jmlr.org/beta/papers/v16/berend15a.html
and
https://projecteuclid.org/euclid.aos/1564797865
.
A: We're looking at the equation $$-\sum_{x\in\mathcal{X}} P(x) \log\left(\frac{Q(x)}{P(x)}\right)
=
-\sum_{x\in\mathcal{X}} Q(x) \log\left(\frac{P(x)}{Q(x)}\right). 
$$
In the Bernoulli case,
$$-p \log\left(\frac{q}{p}\right)
-(1-p) \log\left(\frac{1-q}{1-p}\right)
=
-q \log\left(\frac{p}{q}\right)
-(1-q) \log\left(\frac{1-p}{1-q}\right)
$$
The only solutions are random variables $X$ and $Y$ with $X=1-Y$ (mentioned by @Aryeh) and the trivial solution $X=Y$:

A: I doubt much can be said. One example is where $p$ is a translation of $q$, but there are many others. I will use the notation and results from my answer at https://stats.stackexchange.com/questions/188903/intuition-on-the-kullback-leibler-kl-divergence/189758#189758, where the KL divergence is interpreted in the context of statistical hypothesis testing. 
Let (assuming $P$ and $Q$ are equivalent measures)
$$ \DeclareMathOperator{\KL}{KL}
\KL(P || Q) = \int  \log \frac{dP}{dQ} \; dP
$$ If
$$ \KL(P || Q)=\KL(Q || P) $$ using the interpretation from the link above, $\KL(P || Q)$ is the expected value under the alternative for the log likelihood ratio for the testing problem $H_0 \colon X \sim Q; H_1 \colon X \sim P$, while $\KL(Q || P)$ is the expected value under the alternative for the log likelihood test for the testing problem $H_0 \colon X \sim P; H_1 \colon X \sim Q$. So in this sense, this two testing problems are equally difficult.  
I can see no reason why this should tell us much about the structure of $P$ and $Q$. Maybe one can conclude that $P, Q$ must have similar tail weights (the intuition behind that comment is the normal/t example in the linked post.)
