edu.stanford.nlp.international.arabic.process

Class IOBUtils

java.lang.Object

edu.stanford.nlp.international.arabic.process.IOBUtils

public class IOBUtils
extends java.lang.Object

A class for converting strings to input suitable for processing by an IOB sequence model.

Author:

Spence Green, Will Monroe

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BeginSymbol
static java.lang.String	ContinuationSymbol
static java.lang.String	NosegSymbol
static java.lang.String	RewriteSymbol
static java.lang.String	RewriteTahSymbol Deprecated. use RewriteSymbol instead
static java.lang.String	RewriteTareefSymbol Deprecated. use RewriteSymbol instead

Method Summary

Modifier and Type	Method and Description
static java.lang.String	getBoundaryCharacter()
static java.lang.String	IOBToString(java.util.List<CoreLabel> labeledSequence) Convert a list of labeled characters to a String.
static java.lang.String	IOBToString(java.util.List<CoreLabel> labeledSequence, java.lang.String segmentationMarker) Convert a list of labeled characters to a String.
static java.lang.String	IOBToString(java.util.List<CoreLabel> labeledSequence, java.lang.String prefixMarker, java.lang.String suffixMarker) Convert a list of labeled characters to a String.
static void	labelDomain(java.util.List<CoreLabel> tokenList, java.lang.String domain)
static java.util.List<CoreLabel>	StringToIOB(java.util.List<CoreLabel> tokenList, java.lang.Character segMarker, boolean applyRewriteRules) Convert a String to a list of characters suitable for labeling in an IOB segmentation model.
static java.util.List<CoreLabel>	StringToIOB(java.util.List<CoreLabel> tokenList, java.lang.Character segMarker, boolean applyRewriteRules, boolean stripRewrites) Convert a String to a list of characters suitable for labeling in an IOB segmentation model.
static java.util.List<CoreLabel>	StringToIOB(java.lang.String string) This version is for turning an unsegmented string to an IOB input, i.e., for processing raw text.
static java.util.List<CoreLabel>	StringToIOB(java.lang.String str, java.lang.Character segMarker)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BeginSymbol

public static final java.lang.String BeginSymbol

See Also:

Constant Field Values

ContinuationSymbol

public static final java.lang.String ContinuationSymbol

See Also:

Constant Field Values

NosegSymbol

public static final java.lang.String NosegSymbol

See Also:

Constant Field Values

RewriteSymbol

public static final java.lang.String RewriteSymbol

See Also:

Constant Field Values

RewriteTahSymbol

public static final java.lang.String RewriteTahSymbol

Deprecated. use RewriteSymbol instead

See Also:

Constant Field Values

RewriteTareefSymbol

public static final java.lang.String RewriteTareefSymbol

Deprecated. use RewriteSymbol instead

See Also:

Constant Field Values

Method Detail

getBoundaryCharacter

public static java.lang.String getBoundaryCharacter()

StringToIOB

public static java.util.List<CoreLabel> StringToIOB(java.util.List<CoreLabel> tokenList,
                                                    java.lang.Character segMarker,
                                                    boolean applyRewriteRules)

Convert a String to a list of characters suitable for labeling in an IOB segmentation model.

Parameters:

tokenList -

segMarker -

applyRewriteRules - add rewrite labels (for training data)

StringToIOB

public static java.util.List<CoreLabel> StringToIOB(java.util.List<CoreLabel> tokenList,
                                                    java.lang.Character segMarker,
                                                    boolean applyRewriteRules,
                                                    boolean stripRewrites)

Convert a String to a list of characters suitable for labeling in an IOB segmentation model.

Parameters:

tokenList -

segMarker -

applyRewriteRules - add rewrite labels (for training data)

stripRewrites - revert training data to old Green & DeNero model (remove rewrite labels but still rewrite to try to preserve raw text)

StringToIOB

public static java.util.List<CoreLabel> StringToIOB(java.lang.String string)

This version is for turning an unsegmented string to an IOB input, i.e., for processing raw text.

StringToIOB

public static java.util.List<CoreLabel> StringToIOB(java.lang.String str,
                                                    java.lang.Character segMarker)

IOBToString

public static java.lang.String IOBToString(java.util.List<CoreLabel> labeledSequence,
                                           java.lang.String prefixMarker,
                                           java.lang.String suffixMarker)

Convert a list of labeled characters to a String. Include segmentation markers for prefixes and suffixes in the string, and add a space at segmentations.

IOBToString

public static java.lang.String IOBToString(java.util.List<CoreLabel> labeledSequence,
                                           java.lang.String segmentationMarker)

Convert a list of labeled characters to a String. Include segmentation markers (but no spaces) at segmentation boundaries.

IOBToString

public static java.lang.String IOBToString(java.util.List<CoreLabel> labeledSequence)

Convert a list of labeled characters to a String. Preserve the original (unsegmented) text.

labelDomain

public static void labelDomain(java.util.List<CoreLabel> tokenList,
                               java.lang.String domain)