Validation of Average Error Rate Over Classifiers

Bax, Eric (1997) Validation of Average Error Rate Over Classifiers. Technical Report. California Institute of Technology. [CaltechCSTR:1997.cs-tr-97-17]

Full text available as:

	Postscript - Requires a viewer, such as GhostView
	Other (Adobe PDF (150KB))

Abstract

We examine methods to estimate the average and variance of test error rates over a set of classifiers. We begin with the process of drawing a classifier at random for each example. Given validation data, the average test error rate can be estimated as if validating a single classifier. Given the test example inputs, the variance can be computed exactly. Next, we consider the process of drawing a classifier at random and using it on all examples. Once again, the expected test error rate can be validated as if validating a single classifier. However, the variance must be estimated by validating all classifers, which yields loose or uncertain bounds.

EPrint Type:	Monograph (Technical Report)
Subjects:	All Records
ID Code:	175
Deposited By:	Caltech Library System
Deposited On:	30 April 2001
Record Number:	CaltechCSTR:1997.cs-tr-97-17
Official Persistent URL:	http://resolver.caltech.edu/CaltechCSTR:1997.cs-tr-97-17
Usage Policy:	You are granted permission for individual, educational, research and non-commercial reproduction, distribution, display and performance of this work in any format.

Archive Staff Only: edit this record