After analyze feature importane with SHAP what is next?

Question

I am working on Boosting algorithms. In the first part, i build model, evaluate it (getting satisfied r2) and i look up feature importance with SHAP. Then i found 2 of 5 feature is not big deal.

Then i rearrange my dataset into 3 feature (removing 2 unnecessary features) and refresh this process.

My question is: Should i rearrange my model's hyperparameters ? or should't change anything on model and evaluate on 3 features with same hyperparameters ?

I am asking because in my opinion SHAP tells me these 2 feature is unnecessary on my model which i build with some hyperparameters lets say hyperparameter A. Removing 2 features, should i use for same model hyperparameter A or hyperparameter B which i can find.