edu.stanford.nlp.international.arabic.process

Class IOBUtils

java.lang.Object

edu.stanford.nlp.international.arabic.process.IOBUtils

public class IOBUtils
extends Object

A class for converting strings to input suitable for processing by an IOB sequence model.

Author:

Spence Green, Will Monroe

Field Summary

Fields

Modifier and Type	Field and Description
static String	BeginSymbol
static String	ContinuationSymbol
static String	NosegSymbol
static String	RewriteSymbol
static String	RewriteTahSymbol Deprecated. use RewriteSymbol instead
static String	RewriteTareefSymbol Deprecated. use RewriteSymbol instead

Method Summary

Methods

Modifier and Type	Method and Description
static String	getBoundaryCharacter()
static String	IOBToString(List<CoreLabel> labeledSequence) Convert a list of labeled characters to a String.
static String	IOBToString(List<CoreLabel> labeledSequence, String segmentationMarker) Convert a list of labeled characters to a String.
static String	IOBToString(List<CoreLabel> labeledSequence, String prefixMarker, String suffixMarker) Convert a list of labeled characters to a String.
static void	labelDomain(List<CoreLabel> tokenList, String domain)
static List<CoreLabel>	StringToIOB(List<CoreLabel> tokenList, Character segMarker, boolean applyRewriteRules) Convert a String to a list of characters suitable for labeling in an IOB segmentation model.
static List<CoreLabel>	StringToIOB(List<CoreLabel> tokenList, Character segMarker, boolean applyRewriteRules, boolean stripRewrites) Convert a String to a list of characters suitable for labeling in an IOB segmentation model.
static List<CoreLabel>	StringToIOB(String string) This version is for turning an unsegmented string to an IOB input, i.e., for processing raw text.
static List<CoreLabel>	StringToIOB(String str, Character segMarker)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BeginSymbol

public static final String BeginSymbol

See Also:

Constant Field Values

ContinuationSymbol

public static final String ContinuationSymbol

See Also:

Constant Field Values

NosegSymbol

public static final String NosegSymbol

See Also:

Constant Field Values

RewriteSymbol

public static final String RewriteSymbol

See Also:

Constant Field Values

RewriteTahSymbol

public static final String RewriteTahSymbol

Deprecated. use RewriteSymbol instead

See Also:

Constant Field Values

RewriteTareefSymbol

public static final String RewriteTareefSymbol

Deprecated. use RewriteSymbol instead

See Also:

Constant Field Values

Method Detail

getBoundaryCharacter

public static String getBoundaryCharacter()

StringToIOB

public static List<CoreLabel> StringToIOB(List<CoreLabel> tokenList,
                          Character segMarker,
                          boolean applyRewriteRules)

Convert a String to a list of characters suitable for labeling in an IOB segmentation model.

Parameters:

tokenList -

segMarker -

applyRewriteRules - add rewrite labels (for training data)

StringToIOB

public static List<CoreLabel> StringToIOB(List<CoreLabel> tokenList,
                          Character segMarker,
                          boolean applyRewriteRules,
                          boolean stripRewrites)

Convert a String to a list of characters suitable for labeling in an IOB segmentation model.

Parameters:

tokenList -

segMarker -

applyRewriteRules - add rewrite labels (for training data)

stripRewrites - revert training data to old Green & DeNero model (remove rewrite labels but still rewrite to try to preserve raw text)

StringToIOB

public static List<CoreLabel> StringToIOB(String string)

This version is for turning an unsegmented string to an IOB input, i.e., for processing raw text.

StringToIOB

public static List<CoreLabel> StringToIOB(String str,
                          Character segMarker)

IOBToString

public static String IOBToString(List<CoreLabel> labeledSequence,
                 String prefixMarker,
                 String suffixMarker)

Convert a list of labeled characters to a String. Include segmentation markers for prefixes and suffixes in the string, and add a space at segmentations.

IOBToString

public static String IOBToString(List<CoreLabel> labeledSequence,
                 String segmentationMarker)

Convert a list of labeled characters to a String. Include segmentation markers (but no spaces) at segmentation boundaries.

IOBToString

public static String IOBToString(List<CoreLabel> labeledSequence)

Convert a list of labeled characters to a String. Preserve the original (unsegmented) text.

labelDomain

public static void labelDomain(List<CoreLabel> tokenList,
               String domain)