Here is an ultrafinitist manifesto (link) I have co-written a few years ago:
https://arxiv.org/pdf/cs/0611100v1
- Mirco A. Mannucci, Rose M. Cherubin, Model Theory of Ultrafinitism I: Fuzzy Initial Segments of Arithmetics, https://arxiv.org/abs/cs/0611100
It is absolutely not a finished paper (there are a few inaccuracies, and many parts are sloppy), but it contains a brief history of ultrafinitistic ideas from the early greeks all the way to present time, a number of refs, as well as a sketch of a programme towards a model theory and a proof theory of ultrafinitistic mathematics.
You may also want to google the FOM list on "ultrafinitism", there are a few posts by Podnieks, Sazonov, myself, and a few others pro and contra ultrafinitism.