'Noise-level estimation with scikit-learn GPR package for multidimensional data
I am trying to estimate the noise level for Gaussian Process. Scikit-learn gives an example on this website https://scikit-learn.org/stable/auto_examples/gaussian_process/plot_gpr_noisy.html. The last chunk of code seems to do exactly what I need, but it is only for the 1-d case
from matplotlib.colors import LogNorm
length_scale = np.logspace(-2, 4, num=50)
noise_level = np.logspace(-2, 1, num=50)
length_scale_grid, noise_level_grid = np.meshgrid(length_scale, noise_level)
log_marginal_likelihood = [
gpr.log_marginal_likelihood(theta=np.log([0.36, scale, noise]))
for scale, noise in zip(length_scale_grid.ravel(), noise_level_grid.ravel())
]
log_marginal_likelihood = np.reshape(
log_marginal_likelihood, newshape=noise_level_grid.shape
)
vmin, vmax = (-log_marginal_likelihood).min(), 50
level = np.around(np.logspace(np.log10(vmin), np.log10(vmax), num=50), decimals=1)
plt.contour(
length_scale_grid,
noise_level_grid,
-log_marginal_likelihood,
levels=level,
norm=LogNorm(vmin=vmin, vmax=vmax),
)
plt.colorbar()
plt.xscale("log")
plt.yscale("log")
plt.xlabel("Length-scale")
plt.ylabel("Noise-level")
plt.title("Log-marginal-likelihood")
plt.show()
However, I have inputs X which is two-dimensional, so I have two length scales to optimize. I modified the code as follows:
from matplotlib.colors import LogNorm
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, WhiteKernel
kernel = 1.0 * RBF(length_scale=np.array([1e-1,1e-1])) + WhiteKernel(
determination
noise_level=1e-2, noise_level_bounds=(1e-10, 1e1)
)
gpr = GaussianProcessRegressor(kernel=kernel, alpha=0.0)
gpr.fit(T, y)
res=gpr.get_params
length_scale1 = np.logspace(-2, 4, num=50)
length_scale2 = np.logspace(-2, 4, num=50)
noise_level = np.logspace(-2, 1, num=50)
length_scale_grid1,length_scale_grid2, noise_level_grid = np.meshgrid(length_scale1, length_scale2,noise_level)
log_marginal_likelihood = [
gpr.log_marginal_likelihood(theta=np.log([0.36, scale1, scale2, noise]))
for scale1, scale2, noise in zip(length_scale_grid1.ravel(),length_scale_grid2.ravel(), noise_level_grid.ravel())
]
log_marginal_likelihood = np.reshape(
log_marginal_likelihood, newshape=noise_level_grid.shape
)
vmin, vmax = (-log_marginal_likelihood).min(), 50
level = np.around(np.logspace(np.log10(vmin), np.log10(vmax), num=50), decimals=1)
The result seems wrong since it always gives the largest length scale and noise estimation. I looked up the source code of scikit-learn and it says
theta : array-like of shape (n_kernel_params,)
So my questions are: 1) for 1d case, theta should be of length 2 (one for length scale and one for noise), but why is there a third valur 0.36 in np.log([0.36, scale, noise]))? 2) is my modification correct? if not, how should I change my code for the 2d case?
Thank you very much!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
