This is a summary of my comments above showing that there is such a model of set theory with atoms (ZFA).
If $X$ and $B\colon a\mapsto B_a$ are as in the question, let me call $B$ an assignment function for $X$. The basic observation is that assignment functions do not like automorphisms.
Lemma 1: $\def\zfa{\mathrm{ZFA}}\zfa(F)$ proves that if $B$ is an assignment function for $X$, and $F$ is an automorphism of the universe such that $F(B)=B$, then $F(a)=a$ for every $a\in X$.
Proof: Since $X=\operatorname{dom}(B)$, we have $F(a)\in F(X)=X$. Moreover, $F(B_a)=B_{F(a)}$, hence the restriction $F\restriction B_a$ is a bijection between $B_a$ and $B_{F(a)}$. This implies $a=F(a)$ as $B$ is an assignment function.$\qquad\Box$
Now let us see what this gives for permutation models. I will first briefly recall the construction to fix the notation. We work in a model of ZFA with a set of atoms $A$. We fix a group of permutations $G\le\mathrm{Sym}(A)$, and a normal (i.e., closed under conjugation) filter $F$ of subgroups of $G$ which contains all point stabilizers $G_a=\{g\in G:g(a)=a\}$, where $a\in A$. Using $\in$-induction, every permutation $g\in G$ extends uniquely to an automorphism $\hat g$ of the universe. A set $X$ is symmetric if its stabilizer $G_X=\{g\in G:\hat g(X)=X\}$ is in $F$, and hereditarily symmetric if all elements of its transitive closure are symmetric. The class $M$ of all hereditarily symmetric sets is a transitive model of ZFA, and each $\hat g$ for $g\in G$ is an automorphism of $M$.
We need automorphisms satisfying replacement in Lemma 1. If $M\models\zfa(\hat g)$, then $g=\hat g\restriction A\in M$, and conversely, if $g\in M$, the construction of $\hat g$ by well-founded recursion can be carried out in $M$, so $\hat g$ is definable in $M$ with parameter $g$. Since $\hat f(g)=f\circ g\circ f^{-1}$, the stabilizer of $g$ is just the centralizer $C(g)$. Thus:
Lemma 2: For $g\in G$, $M\models\zfa(\hat g)$ iff $C(g)\in F$.
Note that if the trivial group $1$ is in $F$, then all sets are hereditarily symmetric, so the model trivializes.
Theorem: Let $M$ be a permutation model of ZFA defined using $G$ and $F$ as above. If $C(g)\in F$ for every $g\in G$, and $1\notin F$, then $M\models{}$“$A$ has no assignment function”.
Proof: Assume that $B\in M$ is an assignment function for $A$. On the one hand, $B$ is symmetric, so its stabilizer $G_B$ is in $F$. On the other hand, if $g\in G_B$, then $g=\mathrm{id}$ by Lemmas 1 and 2, thus $G_B=1$, contradicting the assumptions.$\qquad\Box$
Examples of models satisfying the conditions from the theorem are easy to find: for instance, fix a partition of $A$ into pairs, let $G$ be the group of permutations that respect the partition, and let $F$ be the filter generated by point stabilizers. Then $G$ is abelian (a direct product of two-element groups), so the condition on centralizers trivially holds, and $1\notin F$ as long as $A$ is infinite.