edu.stanford.nlp.ie.regexp

Class NumberSequenceClassifier

java.lang.Object

edu.stanford.nlp.ie.AbstractSequenceClassifier<CoreLabel>

edu.stanford.nlp.ie.regexp.NumberSequenceClassifier

All Implemented Interfaces:

Function<String,String>

public class NumberSequenceClassifier
extends AbstractSequenceClassifier<CoreLabel>

A set of deterministic rules for marking certain entities, to add categories and to correct for failures of statistical NER taggers. This is an extremely simple and ungeneralized implementation of AbstractSequenceClassifier that was written for PASCAL RTE. It could profitably be extended and generalized. It marks a NUMBER category based on part-of-speech tags in a deterministic manner. It marks an ORDINAL category based on word form in a deterministic manner. It tags as MONEY currency signs and things tagged CD after a currency sign. It marks a number before a month name as a DATE. It marks as a DATE a word of the form xx/xx/xxxx (where x is a digit from a suitable range). It marks as a TIME a word of the form x(x):xx (where x is a digit). It marks everything else tagged "CD" as a NUMBER, and instances of "and" appearing between CD tags in contexts suggestive of a number. It requires text to be POS-tagged (have the getString(TagAnnotation.class) attribute). Effectively these rules assume that this classifier will be used as a secondary classifier by code such as ClassifierCombiner: it will mark most CD as NUMBER, and it is assumed that something else with higher priority is marking ones that are PERCENT, ADDRESS, etc.

Author:

Christopher Manning, Mihai (integrated with NumberNormalizer, SUTime)

Field Summary

Fields

Modifier and Type	Field and Description
static Pattern	ARMY_TIME_MORNING
static Pattern	CURRENCY_SYMBOL_PATTERN
static Pattern	CURRENCY_WORD_PATTERN
static Pattern	GENERIC_TIME_WORDS
static Pattern	ORDINAL_PATTERN
static Pattern	PERCENT_SYMBOL_PATTERN
static Pattern	PERCENT_WORD_PATTERN
static boolean	USE_SUTIME_DEFAULT
static String	USE_SUTIME_PROPERTY

Fields inherited from class edu.stanford.nlp.ie.AbstractSequenceClassifier

classIndex, CUT_LABEL, featureFactories, flags, knownLCWords, pad, windowSize

Constructor Summary

Constructors

Constructor and Description
NumberSequenceClassifier()
NumberSequenceClassifier(boolean useSUTime)
NumberSequenceClassifier(Properties props, boolean useSUTime, Properties sutimeProps)

Method Summary

Methods

Modifier and Type	Method and Description
static CoreMap	alignSentence(CoreMap sentence) Copies one sentence replicating only information necessary for SUTime
List<CoreLabel>	classify(List<CoreLabel> document) Classify a List of CoreLabels.
List<CoreLabel>	classifyWithGlobalInformation(List<CoreLabel> tokens, CoreMap document, CoreMap sentence) Classify a List of something that extends CoreMap using as additional information whatever is stored in the document and sentence.
static List<CoreLabel>	copyTokens(List<CoreLabel> srcTokens, CoreMap srcSentence) Create a copy of srcTokens, detecting on the fly if character offsets need adjusting
void	finalizeClassification(CoreMap document) Classification is finished for the document.
void	loadClassifier(ObjectInputStream in, Properties props) Load a classifier from the specified input stream.
static void	main(String[] args)
void	printProbsDocument(List<CoreLabel> document)
void	serializeClassifier(String serializePath) Serialize a sequence classifier to a file on the given path.
void	train(Collection<List<CoreLabel>> docs, DocumentReaderAndWriter<CoreLabel> readerAndWriter) Trains a classifier from a Collection of sequences.
static void	transferAnnotations(CoreLabel src, CoreLabel dst) Transfer from src to dst all annotations generated bu SUTime and NumberNormalizer

Methods inherited from class edu.stanford.nlp.ie.AbstractSequenceClassifier

apply, backgroundSymbol, classify, classifyAndWriteAnswers, classifyAndWriteAnswers, classifyAndWriteAnswers, classifyAndWriteAnswers, classifyAndWriteAnswers, classifyAndWriteAnswersKBest, classifyAndWriteAnswersKBest, classifyAndWriteViterbiSearchGraph, classifyFile, classifyFilesAndWriteAnswers, classifyFilesAndWriteAnswers, classifyKBest, classifyRaw, classifySentence, classifySentenceWithGlobalInformation, classifyStdin, classifyStdin, classifyToCharacterOffsets, classifyToString, classifyToString, classifyWithInlineXML, countResults, countResults, countResultsIOB, countResultsIOB2, countResultsSegmenter, defaultReaderAndWriter, dumpFeatures, getSampler, getSequenceModel, getViterbiSearchGraph, labels, loadClassifier, loadClassifier, loadClassifier, loadClassifier, loadClassifier, loadClassifier, loadClassifierNoExceptions, loadClassifierNoExceptions, loadClassifierNoExceptions, loadClassifierNoExceptions, loadClassifierNoExceptions, loadJarClassifier, makeObjectBankFromFile, makeObjectBankFromFile, makeObjectBankFromFiles, makeObjectBankFromFiles, makeObjectBankFromFiles, makeObjectBankFromReader, makeObjectBankFromString, makePlainTextReaderAndWriter, makeReaderAndWriter, plainTextReaderAndWriter, printFeatureLists, printFeatures, printProbs, printProbsDocuments, printResults, reinit, segmentString, segmentString, tallyOneEntityIOB, train, train, train, train, train, train, windowSize, writeAnswers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

USE_SUTIME_DEFAULT

public static final boolean USE_SUTIME_DEFAULT

USE_SUTIME_PROPERTY

public static final String USE_SUTIME_PROPERTY

See Also:

Constant Field Values

CURRENCY_WORD_PATTERN

public static final Pattern CURRENCY_WORD_PATTERN

CURRENCY_SYMBOL_PATTERN

public static final Pattern CURRENCY_SYMBOL_PATTERN

ORDINAL_PATTERN

public static final Pattern ORDINAL_PATTERN

ARMY_TIME_MORNING

public static final Pattern ARMY_TIME_MORNING

GENERIC_TIME_WORDS

public static final Pattern GENERIC_TIME_WORDS

PERCENT_WORD_PATTERN

public static final Pattern PERCENT_WORD_PATTERN

PERCENT_SYMBOL_PATTERN

public static final Pattern PERCENT_SYMBOL_PATTERN

Constructor Detail

NumberSequenceClassifier

public NumberSequenceClassifier()

NumberSequenceClassifier

public NumberSequenceClassifier(boolean useSUTime)

NumberSequenceClassifier

public NumberSequenceClassifier(Properties props,
                        boolean useSUTime,
                        Properties sutimeProps)

Method Detail

classify

public List<CoreLabel> classify(List<CoreLabel> document)

Classify a List of CoreLabels.

Specified by:

classify in class AbstractSequenceClassifier<CoreLabel>

Parameters:

document - A List of CoreLabels.

Returns:

the same List, but with the elements annotated with their answers.

classifyWithGlobalInformation

public List<CoreLabel> classifyWithGlobalInformation(List<CoreLabel> tokens,
                                            CoreMap document,
                                            CoreMap sentence)

Description copied from class: AbstractSequenceClassifier

Classify a List of something that extends CoreMap using as additional information whatever is stored in the document and sentence. This is needed for SUTime (NumberSequenceClassifier), which requires the document date to resolve relative dates.

Specified by:

classifyWithGlobalInformation in class AbstractSequenceClassifier<CoreLabel>

Returns:

Classified version of the input tokenSequence

finalizeClassification

public void finalizeClassification(CoreMap document)

Description copied from class: AbstractSequenceClassifier

Classification is finished for the document. Do any cleanup (if information was stored as part of the document for global classification)

Overrides:

finalizeClassification in class AbstractSequenceClassifier<CoreLabel>

alignSentence

public static CoreMap alignSentence(CoreMap sentence)

Copies one sentence replicating only information necessary for SUTime

Parameters:

sentence -

transferAnnotations

public static void transferAnnotations(CoreLabel src,
                       CoreLabel dst)

Transfer from src to dst all annotations generated bu SUTime and NumberNormalizer

Parameters:

src -

dst -

copyTokens

public static List<CoreLabel> copyTokens(List<CoreLabel> srcTokens,
                         CoreMap srcSentence)

Create a copy of srcTokens, detecting on the fly if character offsets need adjusting

Parameters:

srcTokens -

srcSentence -

train

public void train(Collection<List<CoreLabel>> docs,
         DocumentReaderAndWriter<CoreLabel> readerAndWriter)

Description copied from class: AbstractSequenceClassifier

Trains a classifier from a Collection of sequences. Note that the Collection can be (and usually is) an ObjectBank.

Specified by:

train in class AbstractSequenceClassifier<CoreLabel>

Parameters:

docs - An ObjectBank or a collection of sequences of IN

readerAndWriter - A DocumentReaderAndWriter to use when loading test files

printProbsDocument

public void printProbsDocument(List<CoreLabel> document)

Specified by:

printProbsDocument in class AbstractSequenceClassifier<CoreLabel>

serializeClassifier

public void serializeClassifier(String serializePath)

Description copied from class: AbstractSequenceClassifier

Serialize a sequence classifier to a file on the given path.

Specified by:

serializeClassifier in class AbstractSequenceClassifier<CoreLabel>

Parameters:

serializePath - The path/filename to write the classifier to.

loadClassifier

public void loadClassifier(ObjectInputStream in,
                  Properties props)
                    throws IOException,
                           ClassCastException,
                           ClassNotFoundException

Description copied from class: AbstractSequenceClassifier

Load a classifier from the specified input stream. The classifier is reinitialized from the flags serialized in the classifier.

Specified by:

loadClassifier in class AbstractSequenceClassifier<CoreLabel>

Parameters:

in - The InputStream to load the serialized classifier from

props - This Properties object will be used to update the SeqClassifierFlags which are read from the serialized classifier

Throws:

IOException - If there are problems accessing the input stream

ClassCastException - If there are problems interpreting the serialized data

ClassNotFoundException - If there are problems interpreting the serialized data

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception