First, observe that you can get around the difficulty that you don't know if high means yes or low in the following way. If you really want to ask the question $\varphi$, you should instead ask the question "you will beep high if and only if $\varphi$". If high means yes, then this is the same as asking $\varphi$. But if high means no, then an affirmative answer to $\varphi$ would result in a low note, which means Adam will beep high, and conversely. This transformation therefore ensures that we can in effect know that high means yes. (I mentioned a similar trick in this MO answer about guessing a number, when there can be wrong answers.)
For the lying issue, let me assume that by lying, you mean that Adam first decides whether to lie or tell the truth, and then calculates what a truthful answer would be, and then when telling the truth plays the appropriate tone, but if lying plays the opposite tone. With this interpretation, a similar trick allows us to extract the desired information. Namely, if you want to ask $\psi$, instead ask "you have decided to be truthful for this question iff $\psi$". If Adam decides to be truthful, then this question is answered the same as $\psi$. If he decides to lie, then he calculates what a truthful answer would be, given that he has already decided to lie, which is the opposite of $\psi$, and so he says the opposite of this. In this way, the double negation of the transformation allows us to get the desired information.
Combining the two transformations allows us to get answers to any desired question.
Now, we simply proceed as follows. Since it seems permissible in the world of your question, let us enumerate all the elements of what you call the universe of ordinary mathematics, and ask of each such element whether it is in Adam's set, using the transformations above. In this way, we find out exactly the set of which he is thinking.
The end result is $\beth_\omega$ many questions. This is the optimal in the sense that any smaller bound on the number of questions would be less than $\beth_n$ for some $n$, with only $\beth_{n+1}$ many possible patterns of answers, but there are $\beth_{\omega+1}$ many sets that Adam might be considering.
Incidently, what you call the universe of ordinary mathematics is closely related to what is known in set theory as $V_{\omega+\omega}$, which is a model of the Zermelo axioms of set theory, one of the first axiomatizations of set theory. The $V$ hierarchy begins with $V_0$ being the empty set, and $V_{\alpha+1}=P(V_\alpha)$ and $V_\lambda=\bigcup_{\alpha\lt\lambda} V_\alpha$ for limit ordinals $\lambda$. Your universe is contained within $V_{\omega+\omega}$, but is actually missing huge parts of $V_{\omega+1}$, because you started only with the natural numbers, rather than the hereditary finite sets. For example, the set $\{\ a_k\mid k\in\mathbb{N}\ \}$, where $a_k=\{\{\{\cdots\}\}\}$ has depth $k$, is missing from your universe, but exists in $V_{\omega+1}$. It follows that your world of mathematics does not have the set HF consisting of all hereditary finite sets, or any similar set with unbounded finite depths. From this, it follows that your world does not satisfy some of the very elementary axioms of set theory, which would allow you to construct HF from the natural numbers. For example, the set mentioned above is the result of a very simple induction on finite-depth finte sets.
The fact that $V_{\omega+\omega}$ itself has no sets of size $\beth_\omega$ is precisely what led to the realization that the Zermelo axioms are too weak to prove even that $\beth_\omega$ exists. This realization led directly to the addition of the Replacement axiom to the axioms of set theory, resulting in the theory now known as ZFC.