$\newcommand{\Ext}{\operatorname{Ext}}$$\newcommand{\To}{\longrightarrow}$$\newcommand{\dash}{\text{-}}$$\newcommand{\sSet}{\mathrm{sSet}}$$\newcommand{\ZZ}{\mathbb{Z}}$For convenience, I will denote by $G$ the group $(\ZZ/p)^\times$, and by $X$ the simplicial set $B(\ZZ/p)$ for $p$ a prime. I will prove later in this answer that there are natural weak equivalences $$ X/G \simeq (X\times_G EG)/BG \simeq B(\ZZ/p \rtimes G)/BG \rlap{\qquad\qquad\text{(I)}} $$
The fundamental group, cohomology, and homology of $X/G$
Before I prove formula (I), let me describe what the fundamental group, homology, and cohomology of the space $X/G$ look like. Assume that $p>2$, otherwise $X/G = B(\ZZ/p)$.
The fundamental group: By the van Kampen theorem, the fundamental group of $X/G \simeq B(\ZZ/p \rtimes G)/BG$ is the quotient of $\ZZ/p \rtimes G$ by the normal subgroup generated by $G$. This is easily seen to be trivial, as it is a quotient of $\ZZ/p$ with non-trivial kernel. So $X/G$ is simply connected.
The cohomology: We will make use of the Leray–Serre spectral sequence described by Mariano Suárez-Alvarez in his answer. Here is what we need to apply the conclusion from his answer:
The action of $\alpha\in G$ on $H_1 X = \pi_1 X = \ZZ/p$ is given by multiplication by $\alpha$.
So $\alpha\in G$ acts on $H^2 X = \Ext^1(H_1 X,\ZZ)$ by multiplication by $\alpha$. We have used the universal coefficient theorem and the fact that $H_2 X = H_2 B(\ZZ/p) = 0$.
Therefore, the action of $\alpha\in G$ on $H^\ast X = \ZZ[x]/\langle px \rangle$ (where $x$ is a generator in degree $2$) is by multiplication by $\alpha^n$ on $H^{2n} X$.
Since $G$ is cyclic of order $p-1$, the action of $G$ on $H^{2n} X$ is trivial if and only if $p-1$ divides $n$. In particular, the invariant subalgebra is: $$ (H^\ast X)^G = \ZZ[x^{p-1}]/\langle p x^{p-1} \rangle $$
Now we use the single row, single column spectral sequence for $H^\ast(X\times_G EG)$ that Mariano Suárez-Alvarez obtains in his answer.
Note that $G$ is cyclic of order $p-1$. So the single non-zero row of the spectral sequence is $H^\ast(BG)=\ZZ[y]/\langle (p-1)y \rangle$ for a generator $y$ in degree $2$.
Therefore, the spectral sequence collapses at $E_2$ for degree reasons. The end result of the spectral sequence is then a canonical short exact sequence of graded abelian groups $$ 0 \To \widetilde{H}^\ast BG \To H^\ast(X\times_G EG) \overset{f}{\To} (H^\ast X)^G \To 0 $$ where $f$ is actually an algebra map.
Finally, the long exact sequence for the cohomology of the pair $(X\times_G EG, BG)$ — whose quotient is equivalent to $X/G$ — gives that $H^\ast(X\times_G EG, BG)$ is the kernel of the map $H^\ast(X\times_G EG) \to H^\ast BG$ (which is surjective because it is induced by a section). The previous exact sequence then shows that the composition $$ H^\ast(X/G) = H^\ast((X\times_G EG)/BG) \To H^\ast(X\times_G EG) \To (H^\ast X)^G $$ is an isomorphism. In conclusion, we have an isomorphism of graded algebras: $$ H^\ast(X/G) = H^\ast(X)^G = \ZZ[x^{p-1}]/\langle p x^{p-1} \rangle= \ZZ[u]/\langle pu \rangle $$ where $u=x^{p-1}$ sits in degree $2p-2$. In particular, $H^0(X/G)=\ZZ$, $H^{n(2p-2)}(X/G)\simeq\ZZ/p$ for $n>0$, and all other integral cohomology groups of $X/G$ are zero.
The homology: Observe that $X/G$ is a simplicial set which is finite in each degree. Applying the universal coefficient theorem in reverse, we conclude that $H_0(X/G)=\ZZ$, $H_{n(2p-2)-1}(X/G)\simeq\ZZ/p$ for $n>0$, and all other integral homology groups of $X/G$ are zero.
Proof of the equivalences in formula (I)
As observed in the question, $X$ is a pointed $G$-simplicial set, and the action of $G$ on $X$ is free on every simplex which is not in the basepoint of $X$. This implies that the inclusion of the basepoint $1\to X$ is a cofibration in the projective model structure on $G$-simplicial sets. In other words, $X$ is a cofibrant object in the projective model structure on pointed $G$-simplicial sets.
Now observe that quotienting out by the action of $G$ is a left Quillen adjoint from pointed $G$-simplicial sets to pointed simplicial sets. I will denote this left Quillen adjoint by $F$: $$ F : G\dash\sSet_\ast \To \sSet_\ast $$
Since $X$ is a cofibrant object in the domain of $F$, the quotient $X/G = F(X)$ is weakly equivalent to $LF(X)$, the left derived functor of $F$ applied to $X$:$$ X/G \simeq LF(X) $$ Importantly, $LF(X)$ is weakly equivalent to the quotient of the Borel construction on $X$ by its subspace $BG$: $$ LF(X) \simeq (X\times_G EG)/BG \rlap{\qquad\qquad\text{(#)}} $$ Here, $BG$ includes into the Borel construction via the basepoint of $X$: $$ BG = EG/G = 1\times_G EG \To X\times_G EG $$ At the end of this answer, I will give a proof of formula (#), which simply states the well-known result that the homotopy quotient by $G$ in $\sSet_\ast$ (pointed simplicial sets) is equivalent to the homotopy quotient by $G$ in $\sSet$ further modded out by $BG$.
Finally, as observed in the comments, the Borel construction $X\times_G EG = B(\ZZ/p)\times_G EG$ is actually equivalent to $B(\ZZ/p \rtimes G)$, the classifying space of the semi-direct product. In conclusion: $$ X/G \simeq B(\ZZ/p \rtimes G)/BG $$
Proof of equivalence (#): Consider the sequence of left Quillen adjoints: $$ EG \downarrow G\dash\sSet \overset{T}{\To} 1 \downarrow G\dash\sSet \overset{F}{\To} 1\downarrow\sSet \ (= \sSet_\ast) $$ where the first functor $T$ is left adjoint to pre-composing with $EG\to 1$, i.e. $T$ quotients out by $EG$. Since $EG\to 1$ is a weak equivalence of $G$-simplicial sets, and $G$-simplicial sets form a left proper model category, the functor $T$ is actually a Quillen equivalence. So $X \simeq LT(X)$, where $X$ is seen as an object under $EG$ via $EG\to 1\to X$, and $LT$ is the left derived functor of $T$. Consequently: $$ LF(X) \simeq LF\circ LT(X) \simeq L(F\circ T)(X) \simeq (X\times_G EG)/BG $$ The last weak equivalence is obtained by expressing $F\circ T$ as the composite of the left Quillen adjoints $$ EG\downarrow G\dash\sSet \overset{(-)/G}{\To} BG\downarrow\sSet \overset{(-)/BG}{\To} 1\downarrow\sSet $$ and noting that since $EG$ is cofibrant, the left derived functor of $(-)/G$ is weakly equivalent to the Borel construction.