$\newcommand\R{\mathbb R}$Most likely, the integral $$I:=\int_0^t dt_1 \int_0^t dt_2 \int_0^t dt_3\, B(t-t_3)F(t_3-t_2) S(t-t_2)F(t_2-t_1)F(t_1)$$ cannot be expressed in terms of products and convolutions -- because the finite interval $[0,t]$ is not a group (or even a semigroup), and the restrictions $t_2\in[0,t]$ and $t_3\in[0,t]$ are unjustified, since the integrand contains no factors depending only on $t_2$ or only on $t_3$. Since the functions $B,F,S$ are zero on $(-\infty,0)$, the correct(ed) form of the integral $I$ seems to be \begin{align*} \tilde I&:=\int_\R dt_1 \int_\R dt_2 \int_\R dt_3\, B(t-t_3)F(t_3-t_2) S(t-t_2)F(t_2-t_1)F(t_1) \\ &=\int_\R {dt_2}\, (B*F)(t-t_2)S(t-t_2)(F*F)(t_2) \\ &=\Big(\big((B*F)S\big)*(F*F)\Big)(t), \end{align*} so that $\tilde I$ is expressed in terms of products and convolutions.
The similarly corrected integral added later to the OP, \begin{align*} J&:=\int_\R dt_1 \int_\R dt_2 \int_\R dt_3\, A(t-t_3)F(t_3-t_2)S(t-t_2)F(t_2-t_1)S(t_3-t_1)F(t_1), \end{align*} probably cannot be expressed in terms of products and convolutions.
The reason may be the following shift observation, which addresses a comment by LSpice:
Suppose that a function $f$ can be expressed in terms of products and convolutions of functions $f_1,\dots,f_n$ as follows: \begin{equation} f=\Big(\prod_{j\in J_1}f_j\Big)*\cdots*\Big(\prod_{j\in J_m}f_j\Big), \tag{1} \end{equation} where $(J_1,\dots,J_m)$ is a partition of the set $\{1,\dots,n\}$. Suppose next that, for each $k=1,\dots,m$ and each $j\in J_k$, the function $f_j$ is shifted horizontally by a real number $u_k$. Then $f$ gets shifted by $u_1+\dots+u_k$.
For any given functions $f_1,\dots,f_n$, there may be only finitely many representations of $f$ in the form (1); so, we can try and exhaust all of them.
Using this test, it is easy to see that, e.g., the integral $J$ cannot be expressed as the pure convolution of the functions $A,F,S,F,S,F$ -- take, for instance, each of the functions $A,F,S$ to be the density function of the normal distribution with mean $u$ and variance $1$. Other combinations of the form (1) can be considered similarly.