If there is such a theory, I doubt that an ultrafinitist would like it. The universal set V has to contain itself as an element, and that means you can have chains of unlimited length of the form $x \in y \in \ldots \in z$. But an ultrafinitist doesn't want objects of infinite size, and V smells very much like an object of infinite size (depth).
Depending on the exact interpretation of your axiom #2, and what you mean by set equality, I don't think such a set theory exists. If #2 is meant to say that for any set, there is a singleton containing that set as its member, then we have the existence of $\emptyset$, {$\emptyset$}, {{$\emptyset$}}, ... and all of these are elements of V. If set equality means what people usually take it to mean, then these are all unequal, so V is infinite.
[EDIT: more material added below]
If, on the other hand, the meaning of your axiom #2 is as assumed in JDH's answer, and not as assumed in mine, then there is no generic machinery in your axioms for building new sets out of old ones. This raises the question of the specific meaning of "a certain amount of arithmetic." In JDH's answer, the objects P and Q can be identified with the numbers 1 and 2, and we can clearly make models where this process extends up to some larger integer, such as 387. Then you have a theory with enough objects in it to name the first 387 integers. Is this enough arithmetic? To make this a compelling realization of what we have in mind for a set theory, I think you would want to describe things like "the set of all odd integers less than 153," but your axioms don't have enough machinery in them to generate anything like that.
I think the basic problem here is that if we're going to be ultrafinitists and only admit the existence of integers up to some size $n$, then any set theory that can describe the existence of all sets made out of those integers is going to have a number of sets much, much larger than $n$ --- unless we don't provide enough machinery to generate a rich universe of sets out of these integers, in which case it won't feel like a compelling realization of what we have in mind when we talk about a set theory. This problem is bad enough without the universal set. The universal set just makes it infinitely bad.
