Solving for an operator by minimization Please note that I am looking for numerical algorithms that will tell me what the operator is that minimizes a problem.
I have a 2x2 complex hermitian operator that is a function of two variables, so $\tilde{O}  = \tilde{O}(x,y)$.  I will not have this operator in a closed form, but rather as the output of a numberical simulation who's inputs are just $x$ and $y$.  I want to find out which values of $x$ and $y$ will allow me to map a given vector $\psi_{in}$ to another given vector $\phi_{1}$.  I feel that some kind of gradient descent algorithm is in order.  For instance,  I can define
$d = | \tilde{O}(x,y)\psi_{in} - \phi_{1} |$
Now I can minimize this value over the two input values $x$ and $y$.  So I would have an equation like this:
$ \frac{\partial^2 d}{\partial_x \partial_y} = \frac{\partial^2 | \tilde{O}(x,y)\psi_{in} - \phi_{1} |}{\partial_x \partial_y} $
We might do some minimization from here but I am not sure how to procede and I feel there is a simpler numerical recipe that someone might know of.
Any help will be most appreciated.
 A: Why not use the standard numerical methods for solving a system of equations, available in Maple, Matlab, Mathematica  etc?
A: You haven't said so, but I'm assuming that $\psi$ and $\phi$ are vectors.  These could more generally be functions in some function space, and you would typically discretize those functions to work with vectors in your numerical computations.  
I'm also assuming that you have one or more $\psi$, $\phi$ examples containing measured values or values that have been computed by some numerical model.  Call these examples  $\psi_{1}$, $\phi_{1}$, $\psi_{2}$, $\phi_{2}$, $\ldots$, $\psi_{n}$,$\phi_{n}$.    
There may well be no exact solution to $O(x,y)\psi=\phi$ that works for all of these examples, so a typical approach would be to turn this into a nonlinear least squares problem of the form
$\min_{x,y} \sum_{k=1}^{n} \| O(x,y)\psi_{k}-\phi_{k} \|_{2}^{2} $
Your data and parameters are complex valued, but these are most easily dealt with by splitting the real and imaginary parts to produce a nonlinear least squares problems involving only real parameters and measured values.  
Once you've reduced the problem to a conventional real nonlinear least squares problem, you can use standard methods to solve it.  The Levenberg-Marquadt method is most commonly used in practice.  In the LM method, you can use finite difference approximations to get the required derivatives- most software for doing this has the ability to do this finite differencing for you.  
A: You are interested in what is called direct, or derivative-free optimization. The algorithm can only use a "zero-order oracle" which, when asked, supplies the value of the function at a point. It used to be a not-very-popular subject with a few ad hoc algorithms. The title of a 1996 paper by Margaret Wright, "Direct Search Methods: Once Scorned, Now Respectable" says a lot. It is available at 
http://cm.bell-labs.com/cm/cs/doc/96/4-02.ps.gz
A recent book which gives a readable overview of the field is 
http://www.amazon.com/Introduction-Derivative-Free-Optimization-Mps-Siam-Series/dp/0898716683
Another book that mentions direct optimization is 
http://www.amazon.com/Introductory-Lectures-Convex-Optimization-Applied/dp/1402075537/ref=sr_1_1?s=books&ie=UTF8&qid=1336182876&sr=1-1
