With $5$ you can using the following moves:

```
.....  .....  .....  .....  .....  .....  .....  ..oo.  ...oo 
.....  ..o..  ..o..  ..o..  ..oo.  ...oo  ..ooo  ..ooo  ..ooo 
.oo..  .oo..  .oo..  ..oo.  ..oo.  ..oo.  ..oo.  .....  ..... 
ooo..  oo...  .oo..  ..oo.  ..o..  ..o..  .....  .....  ..... 
```


So the only numbers of pieces for which no configuration can go to infinity are $1$ and $3$.