Signature Construction
Signature construction is the creation of a classifier that maps the feature vector of each observational unit to a set of labels. The term classifier refers to any type of classification method, statistical model, or machine learning algorithm that maps features to labels. Labels can be a discrete set (e.g., Threat, Not a Threat) or a quantitative interval (e.g., predicting the mass of an object). Ideally, each prediction also provides a measure of uncertainty.
There are two steps in constructing the classifier:
- Train the classifier
- Estimate classifier parameters by optimizing one or more objective functions.
- Test the classifier
- Use the random subsets of data not used for training a particular instance of the classifier to test the fidelity of the classifier predictions.
- The measure of classifier fidelity will likely apply principles from "Signature quality assessment" (see below).