edu.stanford.nlp.tagger.maxent
Class TestSentence

java.lang.Object
  edu.stanford.nlp.tagger.maxent.TestSentence

All Implemented Interfaces:

SequenceModel

public class TestSentence
extends java.lang.Object
implements SequenceModel

Author:

Kristina Toutanova, Michel Galley

Field Summary

protected java.lang.String[]	correctTags
protected static boolean	DBG
protected java.lang.String	encoding
protected java.lang.String[]	finalTags
protected static int	kBestSize
protected double[][]	localContextScores
protected java.util.Map<java.lang.String,double[]>	localScores
protected MaxentTagger	maxentTagger
protected static java.lang.String	naTag
protected java.util.List<java.lang.String>	originalTags
protected java.util.List<HasWord>	origWords
protected PairsHolder	pairs
protected java.util.List<java.lang.String>	sent
protected int	size
protected java.lang.String	tagSeparator
protected boolean	VERBOSE

Constructor Summary

TestSentence(MaxentTagger maxentTagger)

TestSentence(MaxentTagger maxentTagger, java.lang.String[] s, java.lang.String[] correctTags, PrintFile pf, Dictionary wrongWords, boolean verboseResults)

Method Summary

protected void	calculateProbs(double[][][] probabilities) calculateProbs puts log probs of taggings in the probabilities array.
protected void	cleanUpScorer() clean-up after the scorer
protected double[]	getHistories(java.lang.String[] tags, History h)
int[]	getPossibleValues(int pos) 0...leftWindow-1 etc are null, leftWindow...length+leftWindow-1 are words, length+leftWindow...length+leftWindow+rightWindow-1 are null;
protected void	init()
protected void	initializeScorer()
int	leftWindow() How many positions to the left a position is dependent on.
int	length()
protected void	revert(int prevSize)
int	rightWindow() How many positions to the right a position is dependent on.
double	scoreOf(int[] sequence) Computes the score assigned by this model to the whole sequence.
double	scoreOf(int[] tags, int pos) Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.
double[]	scoresOf(int[] tags, int pos) Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.
protected java.lang.String[]	stringTagsAt(int pos)
java.util.ArrayList<TaggedWord>	tagSentence(java.util.List<? extends HasWord> s, boolean reuseTags) Tags the sentence s by running maxent model.
protected void	writeTagsAndErrors(java.lang.String[] finalTags, PrintFile pf, Dictionary wrongWords, boolean verboseResults) Write the tagging and note any errors (if pf != null) and accumulate global statistics.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VERBOSE

protected final boolean VERBOSE

naTag

protected static final java.lang.String naTag

See Also:

Constant Field Values

DBG

protected static final boolean DBG

See Also:

Constant Field Values

kBestSize

protected static final int kBestSize

See Also:

Constant Field Values

tagSeparator

protected final java.lang.String tagSeparator

encoding

protected final java.lang.String encoding

pairs

protected final PairsHolder pairs

sent

protected java.util.List<java.lang.String> sent

originalTags

protected java.util.List<java.lang.String> originalTags

origWords

protected java.util.List<HasWord> origWords

size

protected int size

correctTags

protected java.lang.String[] correctTags

finalTags

protected java.lang.String[] finalTags

localScores

protected volatile java.util.Map<java.lang.String,double[]> localScores

localContextScores

protected volatile double[][] localContextScores

maxentTagger

protected final MaxentTagger maxentTagger

Constructor Detail

TestSentence

public TestSentence(MaxentTagger maxentTagger)

TestSentence

public TestSentence(MaxentTagger maxentTagger,
                    java.lang.String[] s,
                    java.lang.String[] correctTags,
                    PrintFile pf,
                    Dictionary wrongWords,
                    boolean verboseResults)

Method Detail

tagSentence

public java.util.ArrayList<TaggedWord> tagSentence(java.util.List<? extends HasWord> s,
                                                   boolean reuseTags)

Tags the sentence s by running maxent model. Returns a sentence (List) of TaggedWord objects.

Parameters:

s - Input sentence (List). This isn't changed.

Returns:

Tagged sentence

revert

protected void revert(int prevSize)

init

protected void init()

calculateProbs

protected void calculateProbs(double[][][] probabilities)

calculateProbs puts log probs of taggings in the probabilities array.

Parameters:

probabilities - Array with indices sent size, k best size, numTags

writeTagsAndErrors

protected void writeTagsAndErrors(java.lang.String[] finalTags,
                                  PrintFile pf,
                                  Dictionary wrongWords,
                                  boolean verboseResults)

Write the tagging and note any errors (if pf != null) and accumulate global statistics.

Parameters:

finalTags - Chosen tags for sentence

pf - File to write tagged output to (can be null, then no output; at present it is non-null iff the debug property is set)

wrongWords - Dictionary to accumulate wrong word counts in (cannot be null)

initializeScorer

protected void initializeScorer()

cleanUpScorer

protected void cleanUpScorer()

clean-up after the scorer

getHistories

protected double[] getHistories(java.lang.String[] tags,
                                History h)

length

public int length()

Specified by:

length in interface SequenceModel

Returns:

the length of the sequences modeled by this SequenceModel

leftWindow

public int leftWindow()

Description copied from interface: SequenceModel

How many positions to the left a position is dependent on.

Specified by:

leftWindow in interface SequenceModel

Returns:

the size of the left window used by this sequence model

rightWindow

public int rightWindow()

Description copied from interface: SequenceModel

How many positions to the right a position is dependent on.

Specified by:

rightWindow in interface SequenceModel

Returns:

the size of the right window used by this sequence model

getPossibleValues

public int[] getPossibleValues(int pos)

Description copied from interface: SequenceModel

0...leftWindow-1 etc are null, leftWindow...length+leftWindow-1 are words, length+leftWindow...length+leftWindow+rightWindow-1 are null;

Specified by:

getPossibleValues in interface SequenceModel

Parameters:

pos - the position

Returns:

the set of possible int values at this position, as an int array

scoreOf

public double scoreOf(int[] tags,
                      int pos)

Description copied from interface: SequenceModel

Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.

Specified by:

scoreOf in interface SequenceModel

Parameters:

tags - the sequence containing the rest of the values to condition on

pos - the position of the element to give a distribution for

Returns:

the log score of the token at the specified position in the sequence

scoreOf

public double scoreOf(int[] sequence)

Description copied from interface: SequenceModel

Computes the score assigned by this model to the whole sequence. Typically this will be an unnormalized probability in log space (since the probabilities are small).

Specified by:

scoreOf in interface SequenceModel

Parameters:

sequence - the sequence to compute a score for

Returns:

the score for the sequence

scoresOf

public double[] scoresOf(int[] tags,
                         int pos)

Description copied from interface: SequenceModel

Computes the unnormalized log conditional distribution over values of the element at position pos in the sequence, conditioned on the values of the elements in all other positions of the provided sequence.

Specified by:

scoresOf in interface SequenceModel

Parameters:

tags - the sequence containing the rest of the values to condition on

pos - the position of the element to give a distribution for

Returns:

the scores of the possible tokens at the specified position in the sequence

stringTagsAt

protected java.lang.String[] stringTagsAt(int pos)