easy.evaluate Namespace Reference

Classes
class	TestResult
class	EvaluationResult
class	Contender

Functions
def	defaultHitStrategy
def	getRelativePath
def	verifyFoundMap
def	getConfusionTable
def	splitRunSet
def	asList
def	crossValidate
def	evaluate
def	joust
def	printEvaluationResults

Function Documentation

def easy.evaluate.asList	(		runset,
			purpose = None
	)

You can pass in an actual cvac.RunSet or a dictionary with
the runset and a classmap, as returned by createRunSet.

def easy.evaluate.crossValidate	(		contender,
			runset,
			folds = 10,
			printVerbose = False
	)

Returns summary statistics tp, fp, tn, fn, recall, trueNegRate,
and a detailed matrix of results with one row for
each fold, and one column each for true positive, false
positive, true negative, and false negative counts

def easy.evaluate.defaultHitStrategy	(		origpur,
			foundLabels,
			foundMap,
			tres
	)

def easy.evaluate.evaluate	(		contender,
			runset,
			printVerbose = False
	)

def easy.evaluate.getConfusionTable	(		results,
			foundMap,
			origMap = None,
			origSet = None,
			HitStrategy = defaultHitStrategy
	)

Determine true and false positives and negatives based on
the purpose of original and found labels.
origMap maps the relative file path of every label to the assigned purpose.
The origMap can be constructed from the original RunSet if it
contained purposes.
Returns TestResult, nores

def easy.evaluate.getRelativePath

(

label

)

def easy.evaluate.joust	(		contenders,
			runset,
			method = 'crossvalidate',
			folds = 10,
			verbose = True
	)

evaluate the contenders on the runset, possibly training
and evaluating with n-fold cross-validation or another method.
The contenders parameter is a list of detectors (or
trainer-detector tuples) that are to be evaluated.

def easy.evaluate.printEvaluationResults

(

results

)

def easy.evaluate.splitRunSet	(		runset_pos,
			runset_neg,
			fold,
			chunksize,
			evalsize,
			rndidx
	)

Take parts of runset_pos and runset_neg and re-combine into
a training set and an evaluation set.  For use by crossValidate().

def easy.evaluate.verifyFoundMap

(

foundMap

)

Verify that the all the purposes in the found map are pos or neg