How does Mutual Information from the Sklearn mutual_info_regression work?

Question

Can someone "explain" ( and possibly where can I access) the code and the logic behind the Sklearn mutual_info_regression? For the classification, I think it works based on the KNN, but for the regression problems, I am having problem understanding how it would work.

Answer 1

Mutual_info_regression is used for feature selection. It works by measuring the mutual information (https://en.wikipedia.org/wiki/Mutual_information) between two random variables or a set of feature vectors (the output in simple terms) and the target (y). Mutual information is a way of measuring one variable's dependence on another by measuring statistical dependency (https://en.wikipedia.org/wiki/Independence_(probability_theory)). It is equal to zero if the two variables and completely independent from each and gets higher as they are more dependent. This can be useful for feature selection. As for source code Sklearn is an open-source library and all of it's code can be found at https://github.com/scikit-learn/scikit-learn

Hope this helped!