In general, it is fairly well established that a method of including derivative information into a a Gaussian Process is to:
- Extend the training vector with derivative information $[y, \delta{y}_1, ... \delta{y}_N]$
- Pad out the covariance matrix with entries to be:
Cov[func, func] Cov[func, deriv]
Cov[deriv, func] Cov[deriv, deriv]
- Use this to predict another output vector $[\hat{y}, \delta{\hat{y}}_1.. \delta{\hat{y}}_N]$ and reshape it to be the function + derivatives
The classic entries of the "derivative kernel" is essentially to differentiate the regular kernel function (e.g. the RBF kernel). The justification is to use the argument that the derivative is a linear operator and thus it will generate another Gaussian Process.
However, it feels like there's a missing dependence on the data itself? Also what about the consistency of the predicted derivatives? By consistency I mean if the elements (obtained by the reshape) will necessarily be the same as say, the finite difference estimate of the predicted energies. Note that another way of looking at the "derivative kernel" is to see it as a special case of the standard multi-output Gaussian Process kernel.
