Let $X\subset\mathbb{P}^N$ be a complex irreducible non-degenerate projective variety.

If $$ p\in \mathrm{Sec}(X):=\overline{\bigcup_{x_1,x_2\in X\atop x_1\neq x_2}\langle x_1, x_2\rangle}, $$ the entry locus $\Sigma_p(X)$ is defined as the closure of the set

$$\{ x\in X\mid \exists x'\in X\quad\text{with}\quad x\neq x'\quad\text{and}\quad p\in\langle x,x'\rangle\}. $$ Notice that, if $p\in \mathrm{Sec}(X)$ is a general point, then every irreducible component of $\Sigma_p(X)$ has dimension $\delta(X):=2\dim(X)+1-\dim(\mathrm{Sec}(X))$.

The variety $X\subset\mathbb{P}^N$ is said to be a quadratic entry locus variety (briefly a $QEL$-variety) if for general point $p\in\mathrm{Sec}(X)$ the entry locus $\Sigma_p(X)$ is a quadric hypersurface (of dimension $\delta(X)$).

Now, I assume that $X\subset\mathbb{P}^N$ is a $QEL$-variety, and for a special point $p_0\in \mathrm{Sec}(X)\setminus X$, $\Sigma_{p_0}(X)$ is a smooth quadric hypersurface of dimension $\delta(X)$ (actually I know this is true for a general point $p_0$ on a divisor of $\mathrm{Sec}(X)$).

Is it true that for general point $p\in\mathrm{Sec}(X)$, the entry locus $\Sigma_p(X)$ is a smooth (quadric hypersurface)?

Thanks in advance.

EDIT: I add a note (but I'm not sure about it). Let $\mathcal{E}(X)$ be the space of all entry locus of $X$ (we have a map $\mathrm{Sec}(X)\setminus X\to \mathcal{E}(X)$, hence $\mathcal{E}(X)$ is irreducible). Let $\mathcal{H}(X)$ be the Hilbert scheme of $\delta$-dimensional quadrics contained in $X$, and let $\mathcal{U}(X)\subseteq\mathcal{H}(X)$ be the open set consisting of the smooth $\delta$-dimensional quadrics. By hypothesis $\mathcal{U}(X)\neq \emptyset$ and we have a natural rational map $$ \phi:\mathcal{E}(X)\dashrightarrow \mathcal{H}(X). $$ Moreover, if $Q\in \mathcal{H}(X)$ and $Q\notin \mathcal{E}(X)$, for every point $z\in \langle Q \rangle \setminus Q$, we have $Q\subsetneq \Sigma_z(X)$ and hence $$\langle Q \rangle\subseteq\{p\in \mathrm{Sec}(X): \Sigma_p(X)\mbox{ is not a $\delta$-dimensional quadric}\}.$$ It follows that $\phi$ is birational and the affirmative answer to my question follows if we show that $\mathcal{H}(X)$ is irriducible.

Is $\mathcal{H}(X)$ irriducible?