As we know that bagging ensemble methods work well with the algorithms that have high variance and, in this concern, the best one is decision tree algorithm. In the following Python recipe, we are going to build bagged decision tree ensemble model by using BaggingClassifier function of sklearn with DecisionTreeClasifier (a classification & regression trees algorithm) on Pima Indians diabetes dataset.
First, import the required packages as follows −
from pandas import read_csv from sklearn.model_selection import KFold from sklearn.model_selection import cross_val_score from sklearn.ensemble import BaggingClassifier from sklearn.tree import DecisionTreeClassifier
Now, we need to load the Pima diabetes dataset as we did in the previous examples −
path = r"C:\pima-indians-diabetes.csv" headernames = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class'] data = read_csv(path, names=headernames) array = data.values X = array[:,0:8] Y = array[:,8]
Next, give the input for 10-fold cross validation as follows −
seed = 7 kfold = KFold(n_splits = 10, random_state = seed) cart = DecisionTreeClassifier()
We need to provide the number of trees we are going to build. Here we are building 150 trees −
num_trees = 150
Next, build the model with the help of following script −
model = BaggingClassifier(base_estimator = cart, n_estimators = num_trees, random_state = seed)
Calculate and print the result as follows −
results = cross_val_score(model, X, Y, cv=kfold) print(results.mean())
The output above shows that we got around 77% accuracy of our bagged decision tree classifier model.