Calculating mutual information for feature selection
Answers
This is a difficult question that may require deep knowledge of the problem domain.
It is possible to automatically select those features in your data that are most useful or most relevant for the problem you are working on. This is a process called feature selection.
In this post you will discover feature selection, the types of methods that you can use and a handy checklist that you can follow the next time that you need to select features for a machine learning model.
Feature selection is also called variable selection or attribute selection.
It is the automatic selection of attributes in your data (such as columns in tabular data) that are most relevant to the predictive modeling problem you are working on.
feature selection… is the process of selecting a subset of relevant features for use in model construction
— Feature Selection, Wikipedia entry.
Feature selection is different from dimensionality reduction. Both methods seek to reduce the number of attributes in the dataset, but a dimensionality reduction method do so by creating new combinations of attributes, where as feature selection methods include and exclude attributes present in the data without changing them.
Examples of dimensionality reduction methods include Principal Component Analysis, Singular Value Decomposition and Sammon’s Mapping.