edu.stanford.nlp.ie.machinereading

Class BasicEntityExtractor

java.lang.Object

edu.stanford.nlp.ie.machinereading.BasicEntityExtractor

All Implemented Interfaces:

Extractor, Serializable

Direct Known Subclasses:

RothEntityExtractor

public class BasicEntityExtractor
extends Object
implements Extractor

Uses parsed files to train classifier and test on data set.

Author:

Andrey Gusev, Mason Smith, David McClosky (mcclosky@stanford.edu)

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected Set<String>	annotationsToSkip
protected EntityMentionFactory	entityMentionFactory
protected String	gazetteerLocation
Logger	logger
protected boolean	useBIO
protected boolean	useNERTags
protected boolean	useSubTypes

Constructor Summary

Constructors

Constructor and Description
BasicEntityExtractor(String gazetteerLocation, boolean useSubTypes, Set<String> annotationsToSkip, boolean useBIO, EntityMentionFactory factory, boolean useNERTags)

Method Summary

Modifier and Type	Method and Description
void	annotate(Annotation doc) Annotate an ExtractionDataSet with entities.
String	getEntityTypeForTag(String tag)
static String	labeledSentenceToString(List<CoreLabel> labeledSentence, boolean printNer) for printing labeled sentence in less verbose manner
static BasicEntityExtractor	load(String path, Class<? extends BasicEntityExtractor> entityClassifier, boolean preferDefaultGazetteer) Loads the model from disk.
void	makeAnnotationFromAllNERTags(CoreMap sentence) Converts NamedEntityTagAnnotation tags into EntityMentions.
void	makeAnnotationFromGivenNERTag(CoreMap sentence, String nerTag, String entityType) Converts NamedEntityTagAnnotation tags into EntityMentions.
void	makeEntityMention(CoreMap sentence, int start, int end, String label, List<EntityMention> entities, int sentCount)
EntityMention	makeEntityMention(CoreMap sentence, int start, int end, String label, String identifier)
static String	makeEntityMentionIdentifier(CoreMap sentence, int sentCount, int entId)
void	postprocessSentence(CoreMap sentence, int sentCount)
void	runTestSet(List<List<CoreLabel>> testSet) This should be called after the classifier has been trained and parseAndTrain has been called to accumulate test set This will return precision,recall and F1 measure
void	save(String path) Serializes this extractor to a file
static void	saveCoNLL(PrintStream os, List<List<CoreLabel>> sentences, boolean alreadyBIO)
static void	saveCoNLLFiles(String dir, Annotation dataset, boolean useSubTypes, boolean alreadyBIO)
void	setAnnotationsToSkip(Set<String> annotationsToSkip)
void	setLoggerLevel(Level level)
void	train(Annotation doc) Trains one extractor model using the given dataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

gazetteerLocation

protected String gazetteerLocation

annotationsToSkip

protected Set<String> annotationsToSkip

useSubTypes

protected boolean useSubTypes

useBIO

protected boolean useBIO

entityMentionFactory

protected EntityMentionFactory entityMentionFactory

logger

public final Logger logger

useNERTags

protected boolean useNERTags

Constructor Detail

BasicEntityExtractor

public BasicEntityExtractor(String gazetteerLocation,
                            boolean useSubTypes,
                            Set<String> annotationsToSkip,
                            boolean useBIO,
                            EntityMentionFactory factory,
                            boolean useNERTags)

Method Detail

annotate

public void annotate(Annotation doc)

Annotate an ExtractionDataSet with entities. This will modify the ExtractionDataSet in place.

Specified by:

annotate in interface Extractor

Parameters:

doc - The dataset to label

getEntityTypeForTag

public String getEntityTypeForTag(String tag)

postprocessSentence

public void postprocessSentence(CoreMap sentence,
                                int sentCount)

makeAnnotationFromGivenNERTag

public void makeAnnotationFromGivenNERTag(CoreMap sentence,
                                          String nerTag,
                                          String entityType)

Converts NamedEntityTagAnnotation tags into EntityMentions. This finds the longest sequence of NamedEntityTagAnnotation tags of the matching type.

Parameters:

sentence - A sentence, ideally annotated with NamedEntityTagAnnotation

nerTag - The name of the NER tag to copy, e.g. "DATE".

entityType - The type of the EntityMention objects created

makeAnnotationFromAllNERTags

public void makeAnnotationFromAllNERTags(CoreMap sentence)

Converts NamedEntityTagAnnotation tags into EntityMentions. This finds the longest sequence of NamedEntityTagAnnotation tags of the matching type.

Parameters:

sentence - A sentence annotated with NamedEntityTagAnnotation

makeEntityMention

public void makeEntityMention(CoreMap sentence,
                              int start,
                              int end,
                              String label,
                              List<EntityMention> entities,
                              int sentCount)

makeEntityMentionIdentifier

public static String makeEntityMentionIdentifier(CoreMap sentence,
                                                 int sentCount,
                                                 int entId)

makeEntityMention

public EntityMention makeEntityMention(CoreMap sentence,
                                       int start,
                                       int end,
                                       String label,
                                       String identifier)

runTestSet

public void runTestSet(List<List<CoreLabel>> testSet)

This should be called after the classifier has been trained and parseAndTrain has been called to accumulate test set This will return precision,recall and F1 measure

setAnnotationsToSkip

public void setAnnotationsToSkip(Set<String> annotationsToSkip)

Parameters:

annotationsToSkip - The type of annotation to skip in assigning answer annotations

train

public void train(Annotation doc)

Description copied from interface: Extractor

Trains one extractor model using the given dataset

Specified by:

train in interface Extractor

Parameters:

doc - dataset to train from (this should already have annotations and will typically be created by a reader)

saveCoNLLFiles

public static void saveCoNLLFiles(String dir,
                                  Annotation dataset,
                                  boolean useSubTypes,
                                  boolean alreadyBIO)
                           throws IOException

Throws:

IOException

saveCoNLL

public static void saveCoNLL(PrintStream os,
                             List<List<CoreLabel>> sentences,
                             boolean alreadyBIO)

load

public static BasicEntityExtractor load(String path,
                                        Class<? extends BasicEntityExtractor> entityClassifier,
                                        boolean preferDefaultGazetteer)
                                 throws ClassCastException,
                                        IOException,
                                        ClassNotFoundException

Loads the model from disk.

Parameters:

path - The location of model that was saved to disk

Throws:

ClassCastException - if model is the wrong format

IOException - if the model file doesn't exist or is otherwise unavailable/incomplete

ClassNotFoundException - this would probably indicate a serious classpath problem

save

public void save(String path)
          throws IOException

Description copied from interface: Extractor

Serializes this extractor to a file

Specified by:

save in interface Extractor

Parameters:

path - where to save the extractor

Throws:

IOException

labeledSentenceToString

public static String labeledSentenceToString(List<CoreLabel> labeledSentence,
                                             boolean printNer)

for printing labeled sentence in less verbose manner

Returns:

string for printing

setLoggerLevel

public void setLoggerLevel(Level level)

Specified by:

setLoggerLevel in interface Extractor