$\newcommand\R{\mathbb R}$Most likely, the integral 
$$I:=\int_0^t dt_1 \int_0^t dt_2 \int_0^t dt_3\, B(t-t_3)F(t_3-t_2)
S(t-t_2)F(t_2-t_1)F(t_1)$$
cannot be expressed in terms of products and convolutions -- because the finite interval $[0,t]$ is not a group (or even a semigroup), and the restrictions $t_2\in[0,t]$ and $t_3\in[0,t]$ are unjustified, since the integrand contains no factors depending only on $t_2$ or only on $t_3$. Since the functions $B,F,S$ are zero on $(-\infty,0)$, the correct(ed) form of the integral $I$ seems to be 
\begin{align*}
	\tilde I&:=\int_\R dt_1 \int_\R dt_2 \int_\R dt_3\, B(t-t_3)F(t_3-t_2)
S(t-t_2)F(t_2-t_1)F(t_1) \\
&=\int_\R {dt_2}\, (B*F)(t-t_2)S(t-t_2)(F*F)(t_2) \\ 
&=\Big(\big((B*F)S\big)*(F*F)\Big)(t),
\end{align*}
so that $\tilde I$ is expressed in terms of products and convolutions.