Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
What is the method for evaluating the performance of the classifier?
There are several methods for estimating the generalization error of a model during training. The estimated error supports the learning algorithm to do model choice; i.e., to discover a model of the right complexity that is not affected by overfitting.
Because the model has been constructed, it can be used in the test set to forecast the class labels of earlier unseen data. It is often useful to measure the performance of the model on the test set because such a measure provides an unbiased estimate of its generalization error. The accuracy or error rate evaluated from the test set can be used to compare the associative performance of multiple classifiers on the equal domain.
There are various methods commonly used to evaluate the performance of a classifier which are as follows −
Holdout Method − In the holdout method, the initial record with labeled instances is partitioned into two disjoint sets, known as the training and the test sets, accordingly. A classification model is induced from the training set and its implementation is computed on the test set.
The efficiency of the classifier can be computed depending on the efficiency of the induced model on the test set. The holdout method has various well-known disadvantages. First, some labeled instances are accessible for training because several data are withheld for testing.
As a result, the induced model cannot be as best as when some labeled examples are utilized for training. Second, the model can be hugely dependent on the structure of the training and test sets.
On the other hand, if the training set is too large, then the estimated accuracy computed from the smaller test set is Iess reliable. Therefore an estimate has a broad confidence interval. Finally, the training and test sets are no higher separate from each other.
Random Subsampling − The holdout method can be repeated multiple times to enhance the computation of a classifier's implementation. This method is called random subsampling.
Let acci be the model accuracy during the ith iteration. The overall accuracy is given by accsub=$\mathrm{\displaystyle\sum\limits_{i=1}^k}$acci/k
Random subsampling encounters several issues associated with the holdout approach because it does not use as much data is applicable for training. It also has no control over the several times each data is used for testing and training. Therefore, some data can be used for training more than others.
Cross-Validation −: An alternative to random subsampling is cross-validation. In this method, each data is used multiple times for training and accurately once for testing. Consider that it can partition the record into two equal-sized subsets. First, it can select one of the subsets for training and the other for testing. It can change the roles of the subsets so that the earlier training set becomes the test set. This method is known as twofold cross-validation.
3K+ Views
