# Machine Learning - Random Forest

It is an extension of bagged decision trees. For individual classifiers, the samples of training dataset are taken with replacement, but the trees are constructed in such a way that reduces the correlation between them. Also, a random subset of features is considered to choose each split point rather than greedily choosing the best split point in construction of each tree.

In the following Python recipe, we are going to build bagged random forest ensemble model by using RandomForestClassifier class of sklearn on Pima Indians diabetes dataset.

First, import the required packages as follows −

from pandas import read_csv
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier


Now, we need to load the Pima diabetes dataset as did in previous examples −

path = r"C:\pima-indians-diabetes.csv"
headernames = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
array = data.values
X = array[:,0:8]
Y = array[:,8]


Next, give the input for 10-fold cross validation as follows −

seed = 7
kfold = KFold(n_splits = 10, random_state = seed)


We need to provide the number of trees we are going to build. Here we are building 150 trees with split points chosen from 5 features −

num_trees = 150
max_features = 5


Next, build the model with the help of following script −

model = RandomForestClassifier(n_estimators = num_trees, max_features = max_features)


Calculate and print the result as follows −

results = cross_val_score(model, X, Y, cv = kfold)
print(results.mean())


Output

0.7629357484620642


The output above shows that we got around 76% accuracy of our bagged random forest classifier model.

machine_learning_with_python_improving_performance_of_ml_models.htm