There are plenty of examples of such pairs of functions.
First, extend your $a$ and $b$ to the whole real line by setting them to $0$
outside the interval $[0,1]$. So your relation becomes
$$\int_{-\infty}^\infty a(y-x)a(y)dy=\int_{-\infty}^\infty b(y-x)b(y)dy.$$
Your condition is that this must hold only for $x\in[0,1]$, but I will construct many examples in which this
will hold for all real $x$. Suppose first that this equation holds for all real $x$.
Now let $f(x)=a(-x),\; g(x)=g(-x)$. Then your equation is equivalent to
$$a\star f=b\star g,$$
where $*$ is the convolution. Taking Fourier transforms and using
$\hat{f}=\hat{a}(-s)$, and similar for $\hat{g}$, we obtain
$$\hat{f}(s)\hat{f}(-s)=\hat{g}(s)\hat{g}(-s),\quad\quad\quad\quad\quad\quad (1)$$
and this is equivalent to the assumption that your equation holds for all real $x$.
Now $\hat{f},\hat{g}$ are entire functions of exponential type, bounded in the lower half-plane (by the Wiener-Paley theorem),
so
$$\hat{f}(s)=e^{cs}\prod_j\left(1-\frac{s}{s_j}\right)e^{s/s_j},$$
and they are determined by their zeros and the exponential factor in front. In equation (1), the exponential factors cancel, so you already have plenty of examples: take $a(x)$ with some small support on $(0,1)$ and let $b$ be a small shift: $b(x)=a(x-\epsilon)$ so that it also has support on $(0,1)$.

But there is much more. Let $Z$ be the set of zeros of $\hat{f}$. Then
zeros of $\hat{f}(s)\hat{f}(-s)$ are $Z\cup(-Z)$, and by decomposing this union in some other way: $Z\cup(-Z)=Z_1\cup(-Z_1)$ you obtain a function $\hat{g}$ for which (1) holds. Typically $\hat{f}$ will have infinitely many zeros, so such decomposition can be done in infinitely many ways, by redistributing, say, finitely many of zeros.

To make sure that all these $\hat{f},\hat{g}$ are really Fourier transforms
of some functions supported on $[-1,0]$ one uses the Wiener-Paley theorem.