(A note: I am going to regard simplicial sets as also defined on the empty ordinal as well, with $X(\emptyset) = *$, which is required for the join formula. This is implicit in your first definition and will remove the need for two extra cases for $d_i$ at the end.)
Regarding the "minor" question. The short explanation is that this follows by decomposing the Hom-set according to the preimage of $c$ and $c'$ in $n$, and observing that each decomposition of $n$ provides an initial choice.
In more category-theoretic language, one way to rewrite the convolution is using the "over" category:
$$
(X \star S)_n = \int^{[n] \to [c] \boxplus [c']} X_c \times S_{c'}
$$
where now the coend is taken over the comma category $n \downarrow \boxplus$ whose objects are triples $([c],[c'],f)$ of a pair of objects of $\Delta$ and a morphism from $[n]$ to their ordinal sum. We note that this comma category decomposes as a disjoint union of categories: each $([c],[c'],f)$ determines a decomposition $[n] = f^{-1} [c] \cup f^{-1} [c']$ into a disjoint union, and morphisms preserve such a decomposition. Therefore,
$$
[n] \downarrow \boxplus \simeq \coprod_{[n] = I \cup I'} (I \downarrow \Delta) \times (I' \downarrow \Delta)
$$
This makes the coend decompose:
$$
(X \star S)_n = \coprod_{[n] = I \cup I'} \int^{I \to [c], I' \to [c']} X_c \times S_{c'}
$$
However, the comma category $(I \downarrow \Delta)\times(I' \downarrow \Delta)$ has an initial object: $I \times I'$ itself. Thus, the coend degenerates down to simply being the value:
$$
(X \star S)_n = \coprod_{[n] = I \cup I'} X_{|I|} \times S_{|I'|}
$$
This is slightly different notation for the second definition of the join that you gave.
Now, as for the boundary formulas.
The definition of $d_i$ is as follows. For each $0 \leq i \leq n$, there is a unique map $d^i:[n-1] \to [n]$ in $\Delta$ whose image is $[n] \setminus \{i\}$: $d^i(x) = x$ for $x < i$, and $d^i(x) = x+1$ for $x \geq i$. The induced map $(X \star S)_n \to (X \star S)_{n-1}$ is the map induced by applying the contravariant functor to $d^i$.
Since $(X \star S)_n$ is a disjoint union of sets, it suffices to show that the formula is correct on $X(I) \times S(I')$ for all decompositions of $[n]$ into $I \cup I'$, where $|I| = j+1$ and $|I'| = k+1$. There are two possibilities: either $i \in I$ when $0 \leq i \leq j$, or $i \in I'$ when $j < i \leq n$.
In either case, the map $[n-1] \to [n] = I \cup I'$ induces, by taking preimages, a unique ordered decomposition $[n-1] = J \cup J'$ of $[n-1]$. If $i \in I$, then $J$ has size $|I| - 1$ and $J'$ is mapped isomorphically to $I'$ by $d^i$. In this case, the map $d^i$ is isomorphic to the map $d^i \boxplus id$ on $[j-1] \boxplus [k] \to [j] \boxplus [k]$. If $i \in I'$, we have the reverse possibility, with $d^i$ isomorphic to $id \boxplus d^{i-j-1}$ (the upper index necessary because inserting the identity at the beginning adds $j+1$ elements to the ordered set at the beginning).
In the case $0 \leq i \leq j$, the induced map
$$
d_i: X(I) \times S(I') \to \coprod_{[n-1] = K \cup K'} X(K) \times S(K')
$$
is therefore the map $d^i X(d^i) \times id: X(I) \times S(I') \to X(J) \times S(J')$, followed by the inclusion into the coproduct. In the case $j < i \leq n$, the map is $id \times d^{i-n-1}$ S(d^{i-n-1})$ followed by inclusion.
This recovers the formula for $d_i$ that you have written down, up to inserting copies of a point $*$ as in the remark at the beginning.
