Grassmannian $Gr(k,n)$ is the set of $k$-dimensional subspace of an $n$-dimensional vector space. What are the Grassmannian in types B, C, D? What are the analog of Plucker coordinates and Plucker relations in these cases? Are there some references about this? Thank you very much.

In type $B$ and $D$ these are orthogonal isotropic Grassmannians $$OGr(k,n) \subset Gr(k,n),$$ that parameterize isotropic $k$-dimensional subspaces in a vector space of dimension $n$ endowed with a non-degenerate quadratic form.

In type $C$ these are symplectic isotropic Grassmannians $$SGr(k,n) \subset Gr(k,n),$$ that parameterize isotropic $k$-dimensional subspaces in a vector space of dimension $n$ endowed with a non-degenerate symplectic form.

What these have in common is that they are of the form $G/P$ for $P$ a maximal parabolic. As such each has a minimal projective embedding of the form $G/P \hookrightarrow \mathbb P(V_\omega)$ where $V_\omega$ is a fundamental representation (the analogue of the Plücker coordinates). The two coordinate rings, of $\mathbb P(V_\omega)$ and $G/P$, are $Sym(V_\omega^*)$ and $\oplus_n V_{n\omega}^*$ respectively. The kernel of the ring map $Sym(V_\omega^*) \twoheadrightarrow \oplus_n V_{n\omega}^*$ is generated in degree $2$ by Ramanathan's theorem (whose proof you can read in the unique book by Brion + Kumar), i.e. the analogue of the Plücker relations is the complement to $V_{2\omega}^*$ inside $Sym^2(V_\omega^*)$. I don't know enough about that representation theory in the specific $BCD$ cases to tell you more.

It might also be worth mentioning that the Type A Grassmannians are *minuscule* varieties, meaning they are $G/P$ for a maximal parabolic $P$ corresponding to a minuscule node of the Dynkin diagram. Minuscule (and also cominuscule) varieties tend to behave a bit better than arbitrary homogeneous spaces $G/P$. At least, their combinatorics can be described quite explicitly, like with the Grassmannian: see e.g. https://arxiv.org/abs/math/0608276 or https://arxiv.org/abs/1306.5419. Note that outside of Type A not so many of the nodes of a Dynkin diagram are minuscule.