edu.stanford.nlp.pipeline

Class CleanXmlAnnotator

java.lang.Object

edu.stanford.nlp.pipeline.CleanXmlAnnotator

All Implemented Interfaces:

Annotator

public class CleanXmlAnnotator
extends Object
implements Annotator

An annotator which removes all xml tags (as identified by the tokenizer) and possibly selectively keeps the text between them. Can also add sentence ending markers depending on the xml tag.

Author:

John Bauer, Angel Chang

Nested Class Summary

Nested classes/interfaces inherited from interface edu.stanford.nlp.pipeline.Annotator

Annotator.Requirement

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DEFAULT_ALLOW_FLAWS
static String	DEFAULT_DATE_TAGS
static String	DEFAULT_DOC_ANNOTATIONS_PATTERNS
static String	DEFAULT_DOCID_TAGS
static String	DEFAULT_DOCTYPE_TAGS
static String	DEFAULT_SECTION_ANNOTATIONS_PATTERNS
static String	DEFAULT_SECTION_TAGS
static String	DEFAULT_SENTENCE_ENDERS
static String	DEFAULT_SINGLE_SENTENCE_TAGS
static String	DEFAULT_SPEAKER_TAGS
static String	DEFAULT_TOKEN_ANNOTATIONS_PATTERNS
static String	DEFAULT_UTTERANCE_TURN_TAGS
static String	DEFAULT_XML_TAGS

Fields inherited from interface edu.stanford.nlp.pipeline.Annotator

BINARIZED_TREES_REQUIREMENT, CLEAN_XML_REQUIREMENT, DETERMINISTIC_COREF_REQUIREMENT, GENDER_REQUIREMENT, GUTIME_REQUIREMENT, HEIDELTIME_REQUIREMENT, LEMMA_REQUIREMENT, NER_REQUIREMENT, NUMBER_REQUIREMENT, PARSE_AND_TAG, PARSE_REQUIREMENT, PARSE_TAG_BINARIZED_TREES, POS_REQUIREMENT, QUANTIFIABLE_ENTITY_NORMALIZATION_REQUIREMENT, RELATION_EXTRACTOR_REQUIREMENT, SSPLIT_REQUIREMENT, STANFORD_CLEAN_XML, STANFORD_DETERMINISTIC_COREF, STANFORD_GENDER, STANFORD_LEMMA, STANFORD_NER, STANFORD_PARSE, STANFORD_POS, STANFORD_REGEXNER, STANFORD_RELATION, STANFORD_SENTIMENT, STANFORD_SSPLIT, STANFORD_TOKENIZE, STANFORD_TRUECASE, STEM_REQUIREMENT, SUTIME_REQUIREMENT, TIME_WORDS_REQUIREMENT, TOKENIZE_AND_SSPLIT, TOKENIZE_REQUIREMENT, TOKENIZE_SSPLIT_NER, TOKENIZE_SSPLIT_PARSE, TOKENIZE_SSPLIT_PARSE_NER, TOKENIZE_SSPLIT_POS, TOKENIZE_SSPLIT_POS_LEMMA, TRUECASE_REQUIREMENT

Constructor Summary

Constructors

Constructor and Description
CleanXmlAnnotator()
CleanXmlAnnotator(String xmlTagsToRemove, String sentenceEndingTags, String dateTags, boolean allowFlawedXml)

Method Summary

Methods

Modifier and Type	Method and Description
void	annotate(Annotation annotation) Given an Annotation, perform a task on this Annotation.
List<CoreLabel>	process(Annotation annotation, List<CoreLabel> tokens)
List<CoreLabel>	process(List<CoreLabel> tokens)
Set<Annotator.Requirement>	requirementsSatisfied() Returns a set of requirements for which tasks this annotator can provide.
Set<Annotator.Requirement>	requires() Returns the set of tasks which this annotator requires in order to perform.
void	setDiscourseTags(String utteranceTurnTags, String speakerTags)
void	setDocAnnotationPatterns(String conf)
void	setDocIdTagMatcher(String docIdTags)
void	setDocTypeTagMatcher(String docTypeTags)
void	setSectionAnnotationPatterns(String conf)
void	setSectionTagMatcher(String sectionTags)
void	setSingleSentenceTagMatcher(String tags)
void	setSsplitDiscardTokensMatcher(String tags)
void	setTokenAnnotationPatterns(String conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_XML_TAGS

public static final String DEFAULT_XML_TAGS

See Also:

Constant Field Values

DEFAULT_SENTENCE_ENDERS

public static final String DEFAULT_SENTENCE_ENDERS

See Also:

Constant Field Values

DEFAULT_SINGLE_SENTENCE_TAGS

public static final String DEFAULT_SINGLE_SENTENCE_TAGS

DEFAULT_DATE_TAGS

public static final String DEFAULT_DATE_TAGS

See Also:

Constant Field Values

DEFAULT_DOCID_TAGS

public static final String DEFAULT_DOCID_TAGS

See Also:

Constant Field Values

DEFAULT_DOCTYPE_TAGS

public static final String DEFAULT_DOCTYPE_TAGS

See Also:

Constant Field Values

DEFAULT_UTTERANCE_TURN_TAGS

public static final String DEFAULT_UTTERANCE_TURN_TAGS

See Also:

Constant Field Values

DEFAULT_SPEAKER_TAGS

public static final String DEFAULT_SPEAKER_TAGS

See Also:

Constant Field Values

DEFAULT_DOC_ANNOTATIONS_PATTERNS

public static final String DEFAULT_DOC_ANNOTATIONS_PATTERNS

See Also:

Constant Field Values

DEFAULT_TOKEN_ANNOTATIONS_PATTERNS

public static final String DEFAULT_TOKEN_ANNOTATIONS_PATTERNS

DEFAULT_SECTION_TAGS

public static final String DEFAULT_SECTION_TAGS

DEFAULT_SECTION_ANNOTATIONS_PATTERNS

public static final String DEFAULT_SECTION_ANNOTATIONS_PATTERNS

DEFAULT_ALLOW_FLAWS

public static final boolean DEFAULT_ALLOW_FLAWS

See Also:

Constant Field Values

Constructor Detail

CleanXmlAnnotator

public CleanXmlAnnotator()

CleanXmlAnnotator

public CleanXmlAnnotator(String xmlTagsToRemove,
                 String sentenceEndingTags,
                 String dateTags,
                 boolean allowFlawedXml)

Method Detail

setSsplitDiscardTokensMatcher

public void setSsplitDiscardTokensMatcher(String tags)

setSingleSentenceTagMatcher

public void setSingleSentenceTagMatcher(String tags)

setDocIdTagMatcher

public void setDocIdTagMatcher(String docIdTags)

setDocTypeTagMatcher

public void setDocTypeTagMatcher(String docTypeTags)

setSectionTagMatcher

public void setSectionTagMatcher(String sectionTags)

setDiscourseTags

public void setDiscourseTags(String utteranceTurnTags,
                    String speakerTags)

setDocAnnotationPatterns

public void setDocAnnotationPatterns(String conf)

setTokenAnnotationPatterns

public void setTokenAnnotationPatterns(String conf)

setSectionAnnotationPatterns

public void setSectionAnnotationPatterns(String conf)

annotate

public void annotate(Annotation annotation)

Description copied from interface: Annotator

Given an Annotation, perform a task on this Annotation.

Specified by:

annotate in interface Annotator

process

public List<CoreLabel> process(List<CoreLabel> tokens)

process

public List<CoreLabel> process(Annotation annotation,
                      List<CoreLabel> tokens)

requires

public Set<Annotator.Requirement> requires()

Description copied from interface: Annotator

Returns the set of tasks which this annotator requires in order to perform. For example, the POS annotator will return "tokenize", "ssplit".

Specified by:

requires in interface Annotator

requirementsSatisfied

public Set<Annotator.Requirement> requirementsSatisfied()

Description copied from interface: Annotator

Returns a set of requirements for which tasks this annotator can provide. For example, the POS annotator will return "pos".

Specified by:

requirementsSatisfied in interface Annotator