Are these really all the constraints? Let $u$ be the vector of all $1$'s. Consider solutions of the following form: $p$ is an arbitrary vector with $u^T p = 1$,
$A = B + c u^T$ where $B p = 0$. If your problem has an optimal solution, $f(B)^T p$ must be $0$ for all $B$ such that $B p = 0$.
By scaling and continuity, $f(B)^T x = 0$ for all matrices $B$ and vectors $x$ such that $B x = 0$. This can only happen if $f(B)^T$ is in the span of the rows of $B$, i.e. $f(B) = B^T q$
for some vector $q$. At first sight it appears that $q$ could depend on $B$, but in fact it's not hard to see that if $f$ is linear $q$ must be constant. But then for any $A$ and $p$ with $A p = c$, $f(A)^T p = q^T A p = q^T c$. So the objective is constant on the set of feasible solutions.