'Hyperparameter tuning in KNN

I am trying to find the best k for my model and tries gridsearch cv and ended up having K=1, but k=1 is not a best k most of the times as it wont perform well on test data.

Find the code below.

from sklearn import neighbors
from sklearn import metrics
from sklearn import model_selection
import matplotlib.pyplot as plt

knn_parameters = {'n_neighbors': \[1, 3, 5, 7, 11\],'weights':\['uniform','distance'\],'metric':\['euclidean','manhattan'\]}

knn = neighbors.KNeighborsClassifier()

features = df2.iloc\[:,:-1\]
labels = df2.target

# Train - Test Split

train_features, test_features, train_labels, test_labels = model_selection.train_test_split(features, labels, test_size=0.3, random_state=42)

\#kNN
knn_best_k = model_selection.GridSearchCV(knn, knn_parameters)

knn_best_k.fit( train_features, train_labels)

print("The best classifier for kNN is:", knn_best_k.best_estimator\_)
print("kNN accuracy is:",knn_best_k.best_score\_)
print("kNN parameters are:",knn_best_k.best_params\_)`

Was expecting optimal k but found k = 1.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source