Studies of neural networks that are more general than $\mathbb{R}^n\mapsto\mathbb{R}^k$ include
- Deep Complex Networks (2017)
- Deep Quaternion Networks (2017)
- P-Adic Neural Networks (2004)
- Deep Learning on Lie Groups (2017)
A general overview is provided in Neural Networks on Groups (2019).
One general thing to keep in mind in this context is that the training of the network by steepest descent will rely on continuous differentiability.