In feature engineering, what is a correct method for selecting the best k features with different types?

Question

I have 81 features, and I want to select the best K from it, which will be the most useful for predicting the labels. Some of the features are numerical, some of them are ordinal, and the rest of them are categorical. In this case for example, should I handle different types of feature groups independently?

For example: select all the numerical features then make a correlation matrix, then select those features which has the best correlation with the label (and the least correlation with each other). And next, get the categorical features, do a chi square test, and select the top k attributes. Then other feature selecting method for ordinal variables.

Finally, I would get the top features from each group, individually. Could this be a proper approach for large datasets? Or should I use some kind of embedded approaches, like random forest on the whole dataset to determine the k most important feature?

Answer 1

I would recommend one of the following methods: Sequential Feature Selector (SFS), Sequential Backward Selector (SBS), Sequential Forward floating selector (SFFS),Sequential Backward floating selector (SBFS).

You can read more about it in this link.