I would like to minimize the following objective function
$$ \| H \ast A - (H \cdot I) \ast B \|_F^2 $$
w.r.t. $H$, where
- $H$, $I$, $A$, and $B$ are all square matrices of the same size ($I$ is a flipped identity matrix i.e., off-diagonal elements are 1 and others 0),
- $\ast$ denotes convolution and
- $\cdot$ is the matrix multiplication.
I would like to ask: are there any common routines to solve a problem involving convolution? Also, is this problem convex?
Edit:
I first considered to apply FFT on both $H \ast A$ and $ (H \cdot I) \ast B$. This however still cannot eliminate all convolution operations from the objective.