Your first calculation for $3\times3$ matrices applies in full generality: If your matrix writes blockwise $[1 \quad e^T ; e \quad M]$, with $e^T=(1,\ldots,1)$, and if $M$ is non-singular (implied by your assumption), then the property that $\det A=0$ is equivalent to $e^T\hat M e=\det M$, that is $e^TM^{-1}e=1$, or to $\det(J-M)=0$, with $J=ee^T$ the matrix with $1$s everywhere. Just use the Sherman-Morrison formula $\det(B+xy^T)=(\det B)(1+y^tB^{-1}x)$.
Edit after 3 hours. Take the $3\times3$ matrix $N:=[1 \quad i \quad -1;i \quad -1 \quad1;-1 \quad1 \quad-1]$. We have $N^{-1}e=(-i-1,-2i,-i)^T$. Let $D$ be a diagonal matrix with unit entries on the diagonal, so that $M:=ND^{-1}$ is still an admissible matrix. Then $$e^TM^{-1}e=e^TDN^{-1}e=(-i-1)z_1-2iz_2-iz_3.$$ Claim: There exist unit numbers $z_j$ such that the right-hand side equals $1$. There exists actually a lot of them Consequence: the matrix $A$ is singular. Yet it does not have two equal rows or columns. Proof of the claim: we may search for unit numbers $y_j$ such that $\sqrt2 y_1+2y_2+y_3=1$. Taking $y_3=y_1$, we just have $(1+\sqrt2)y+2y'$, which covers a corona $(\sqrt2-1)\le |z|\le 3+\sqrt2$. In particular the equation $(1+\sqrt2)y+2y'=1$ has a solution. If instead we choose $y_3=e^{i\epsilon}y_1$ with a small enough $\epsilon$, the number $\sqrt2 y_1+y_3$ covers a circle of raidus $\rho$ close to $\sqrt2-1$, and the corona obtained by adding $2y_2$ still contain $1$.