How to output mean and stdv of Gaussian Process Classifier in sklearn?

Question

Im fitting some data for a classification task using Gaussian Process Classifiers in sklearn. I know that for the Gaussian Process Regressor one can pass return_std in

y_test, std = gp.predict(x_test, return_std=True)

to output the standard deviation of the test sample (like in this question)

However, I couldn't find such a parameter for the GP Classifier.

Is there such thing as outputting the predictive mean and stdv of test data from a GP Classifiers? And is there a way to output the posterior mean and covariance of the fitted model?

Answer 1

There is not standard deviation for categorical data, hence there is no the parameter return_std in the Classifier.

However, if you want to quantify the uncertainty of the classifier predictions, you could use the .predict_proba(X)method. Once you get the probabilites of each posible class you could compute the entropy of the predicted probabilities.

Answer 2

You could get the variance associated with the logit function by going to the predict_proba function definition in _gpc.py and returning the 'var_f_star' value. I have modified the predict_proba and created a function to return the logit variance below:

    def predict_var(self, X):
    """Return probability estimates for the test vector X.

    Parameters
    ----------
    X : array-like of shape (n_samples, n_features) or list of object
        Query points where the GP is evaluated for classification.

    Returns
    -------
    C : array-like of shape (n_samples, n_classes)
        Returns the probability of the samples for each class in
        the model. The columns correspond to the classes in sorted
        order, as they appear in the attribute ``classes_``.
    """
    check_is_fitted(self)

    # Based on Algorithm 3.2 of GPML
    K_star = self.kernel_(self.X_train_, X)  # K_star =k(x_star)
    f_star = K_star.T.dot(self.y_train_ - self.pi_)  # Line 4
    v = solve(self.L_, self.W_sr_[:, np.newaxis] * K_star)  # Line 5
    # Line 6 (compute np.diag(v.T.dot(v)) via einsum)
    var_f_star = self.kernel_.diag(X) - np.einsum("ij,ij->j", v, v)