Okay, I've found the relevant notation in Higher Topos Theory. The first is at the end of 1.1.5.10, and is the simplicial set of maps from an n-1 element set to the 1-simplex (i.e., an n-1-cube). The second is at the end of warning 1.2.2.2, and describes a constant map of simplicial sets whose image is x. The curly braces give a reference to the specific ordered set that defines the the simplex.
A simplex at a vertex is a degenerate simplex. You take the simplicial subset defined by x, and demand that the map from your simplex to the target factor through the inclusion of x.
Edit in response to comment: You can think of vertices in (at least) two ways. One way is as an element of S0, i.e., a zero-simplex of the simplicial set. Another way is as a simplicial subset X of S, such that X0 is the chosen element of S0, and all Xi have a single element, namely the image of X0 under the unique degeneracy map. The statement is that the map Z takes a particular nondegenerate n-dimensional face of $\Delta^{n+1}$ to the unique element of Xn.