What are Random Forests?

Random forest is a class of ensemble approaches particularly designed for decision tree classifiers. It integrates the predictions made by several decision trees, where each tree is created based on the values of a separate set of random vectors.

The random vectors are produced from a constant probability distribution, unlike the adaptive methods used in AdaBoost, where the probability distribution is diverse to target instances that are difficult to classify.

Bagging needs decision trees is a definite case of random forests, where randomness is inserted into the model-building procedure by randomly selecting N samples, with restoration, from the initial training set. Bagging also needs a similar uniform probability distribution to make its bootstrapped samples throughout the complete model-building phase.

Each decision tree needs a random vector that is produced from some constant probability distribution. A random vector can be integrated into the tree-growing procedure in several ways. The first method is to randomly choose F input features to divide at each node of the decision tree.

As a result, rather than examining all the accessible features, the decision to divide a node is decided from these selected features. The tree is developed to its entirety without some pruning. This can help decrease the bias present in the outcoming tree.

Because the trees have been built, the predictions are connected using a majority voting design. This approach is called Forest-Rl, where RI defines random input selection. It can improve randomness, bagging can be used to create bootstrap samples for Forest-RI.

The durability and correlation of random forests can be based on the size of F. If F is adequately small, therefore the trees influence becomes less correlated. In other terms, the strength of the tree classifier influence to enhance with a higher number of features, F.

If the multiple original features d is too small, therefore it is complex to select a separate set of random features for constructing the decision trees. There is one method to increase the feature space is to make a linear set of the input features. Particularly, at each node, a new feature is made by randomly choosing the L of the input features.

The input features are linearly linked using coefficients created from a uniform distribution in the range of [-1, 1]. At every node, F of such randomly combined new features are made, and the best of them is finally chosen to divide the node. This approach is called Forest-RC.