Decision tree implementation using Python

PythonServer Side ProgrammingProgramming

Decision tree is an algorithm which is mainly applied to data classification scenarios. It is a tree structure where each node represents the features and each edge represents the decision taken. Starting from the root node we go on evaluating the features for classification and take a decision to follow a specific edge. Whenever a new data point comes in , this same method is applied again and again and then the final conclusion is taken when all the required features are studied or applied to the classification scenario. So Decision tree algorithm is a supervised learning model used in predicting a dependent variable with a series of training variables.

Example

We will take the drug test data available at kaggle. As a first step we will read the data from a csv file using pandas and see it content and structure.

import pandas as pd

datainput = pd.read_csv("drug.csv", delimiter=",") #https://www.kaggle.com/gangliu/drugsets
print(datainput)

Running the above code gives us the following result:

   Age Sex BP   Cholesterol Na_to_K  Drug
0  23   F HIGH  HIGH        25.355   drugY
1  47   M LOW   HIGH        13.093   drugC
2  47   M LOW   HIGH        10.114  drugC
3  28   F NORMAL HIGH        7.798  drugX
4  61   F LOW    HIGH       18.043  drugY
.. ... .. ... ... ... ...
195 56  F LOW    HIGH       11.567  drugC
196 16  M LOW    HIGH       12.006  drugC
197 52  M NORMAL HIGH     9.894 drugX
[200 rows x 6 columns]

Pre-processing the data

In the next step we pre-process the above data to get numeric values for different text values we have in the data. That is useful to train and test the sample data about the decision to use certain drug for a given value of age, sex, BP etc.

Example

import numpy as np
import pandas as pd
from sklearn.metrics import confusion_matrix

datainput = pd.read_csv("drug.csv", delimiter=",")

X = datainput[['Age', 'Sex', 'BP', 'Cholesterol', 'Na_to_K']].values

from sklearn import preprocessing
label_gender = preprocessing.LabelEncoder()
label_gender.fit(['F','M'])
X[:,1] = label_gender.transform(X[:,1])

label_BP = preprocessing.LabelEncoder()
label_BP.fit([ 'LOW', 'NORMAL', 'HIGH'])
X[:,2] = label_BP.transform(X[:,2])

label_Chol = preprocessing.LabelEncoder()
label_Chol.fit([ 'NORMAL', 'HIGH'])
X[:,3] = label_Chol.transform(X[:,3])

# Printing the first 6 records
print(X[0:6])

Running the above code gives us the following result -

[[23 0 0 0 25.355]
   [47 1 1 0 13.093]
   [47 1 1 0 10.113999999999999]
   [28 0 2 0 7.797999999999999]
   [61 0 1 0 18.043]
   [22 0 2 0 8.607000000000001]
]

Converting the Dependent variable

Next we also convert the dependent variable into numerical values so that it can be used in the training as well as the evaluation data set.

Example

import pandas as pd

datainput = pd.read_csv("drug.csv", delimiter=",")
X = datainput[['Age', 'Sex', 'BP', 'Cholesterol', 'Na_to_K']].values

y = datainput["Drug"]

print(y[0:6])

Output

Running the above code gives us the following result:

0    drugY
1    drugC
2    drugC
3    drugX
4    drugY
5    drugX
Name: Drug, dtype: object

Training the Dataset

Next we use 30 percent of the supplied data as a training data set. This will be use as the basis for creating the classification for the remaining 70 percentages which we will call as test data.

Example

import pandas as pd
from sklearn.model_selection import train_test_split

datainput = pd.read_csv("drug.csv", delimiter=",")

X = datainput[['Age', 'Sex', 'BP', 'Cholesterol', 'Na_to_K']].values

y = datainput["Drug"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=3)

print(X_train.shape)
print(X_test.shape)
print(y_train.shape)
print(y_test.shape)

Output

Running the above code gives us the following result:

(140, 5)
(60, 5)
(140,)
(60,)

Getting the result from Trained Data set

Next we can apply the decision tree to see the result for the trained data set. Here we create a tree based on the input we have and using a criteria called entropy. And finally we calculate the accuracy of the decision tree.

Example

import pandas as pd
from sklearn.tree import DecisionTreeClassifier
from sklearn import metrics

datainput = pd.read_csv("drug.csv", delimiter=",")

X = datainput[['Age', 'Sex', 'BP', 'Cholesterol', 'Na_to_K']].values

# Data Preprocessing
from sklearn import preprocessing

label_gender = preprocessing.LabelEncoder()
label_gender.fit(['F', 'M'])
X[:, 1] = label_gender.transform(X[:, 1])

label_BP = preprocessing.LabelEncoder()
label_BP.fit(['LOW', 'NORMAL', 'HIGH'])
X[:, 2] = label_BP.transform(X[:, 2])

label_Chol = preprocessing.LabelEncoder()
label_Chol.fit(['NORMAL', 'HIGH'])
X[:, 3] = label_Chol.transform(X[:, 3])

y = datainput["Drug"]

# train_test_split
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=3)

drugTree = DecisionTreeClassifier(criterion="entropy", max_depth=4)

drugTree.fit(X_train, y_train)
predicted = drugTree.predict(X_test)

print(predicted)

print("\nDecisionTrees's Accuracy: ", metrics.accuracy_score(y_test, predicted))

Output

Running the above code gives us the following result:

['drugY' 'drugX' 'drugX' 'drugX' 'drugX' 'drugC' 'drugY' 'drugA' 'drugB'
'drugA' 'drugY' 'drugA' 'drugY' 'drugY' 'drugX' 'drugY' 'drugX' 'drugX'
'drugB' 'drugX' 'drugX' 'drugY' 'drugY' 'drugY' 'drugX' 'drugB' 'drugY'
'drugY' 'drugA' 'drugX' 'drugB' 'drugC' 'drugC' 'drugX' 'drugX' 'drugC'
'drugY' 'drugX' 'drugX' 'drugX' 'drugA' 'drugY' 'drugC' 'drugY' 'drugA'
'drugY' 'drugY' 'drugY' 'drugY' 'drugY' 'drugB' 'drugX' 'drugY' 'drugX'
'drugY' 'drugY' 'drugA' 'drugX' 'drugY' 'drugX']

DecisionTrees's Accuracy: 0.9833333333333333
raja
Published on 02-Jan-2020 09:57:04
Advertisements