edu.stanford.nlp.ling.tokensregex
Class TokenSequencePattern

java.lang.Object
  edu.stanford.nlp.ling.tokensregex.SequencePattern<CoreMap>
      edu.stanford.nlp.ling.tokensregex.TokenSequencePattern

public class TokenSequencePattern
extends SequencePattern<CoreMap>

Token Sequence Pattern for regular expressions for sequences over tokens (as the more general CoreMap). Sequences over tokens can be matched like strings.

To use

   TokenSequencePattern p = TokenSequencePattern.compile("....");
   TokenSequenceMatcher m = p.getMatcher(tokens);
   while (m.find()) ....
 

Supports the following:

• Concatenation: X Y
• Or: X | Y
• And: X & Y
• Groups:
  • capturing: (X) (with numeric group id)
  • capturing: (?$var X) (with group name "$var")
  • noncapturing: (?:X)
  Capturing groups can be retrieved with group id or group variable, as matched string (m.group()) or list of tokens (m.groupNodes()).
  • To retrieve group using id: m.group(id) or m.groupNodes(id)
    NOTE: Capturing groups are indexed from left to right, starting at one. Group zero is the entire matched sequence.
  • To retrieve group using bound variable name: m.group("$var") or m.groupNodes("$var")
  See SequenceMatchResult for more accessor functions to retrieve matches.
• Greedy Quantifiers: X+, X?, X*, X{n,m}, X{n}, X{n,}
• Reluctant Quantifiers: X+?, X??, X*?, X{n,m}?, X{n}?, X{n,}?
• Back references: \captureid
• Value binding for groups: [pattern] => [value]. Value for matched expression can be accessed using m.groupValue()
  
  Example:

  ( one => 1 | two => 2 | three => 3 | ...)

Individual tokens are marked by "[" TOKEN_EXPR "]"
Possible TOKEN_EXPR:

• All specified token attributes match:
  For Strings: { lemma:/.../; tag:"NNP" } = attributes that need to all match
  NOTE: /.../ used for regular expressions, "..." for exact string matches
  For Numbers: { word>=2 }
  NOTE: Relation can be ">=", "<=", ">", "<", or "=="
  Others: { word::IS_NUM } , { word::IS_NIL } or { word::NOT_EXISTS }, { word::NOT_NIL } or { word::EXISTS }
• Short hand for just word/text match: /.../ or "..."
• Negation: !{...}
• Conjunction or Disjunction: {...} & {...} or {...} | {...}

Special tokens: Any token: []

String pattern match across multiple tokens: (?m){min,max} /pattern/

Binding of variables for use in compiling patterns:

1. Use Env env = TokenSequencePattern.getNewEnv() to create new environment for binding
2. Bind string to attribute key (Class) lookup env.bind("numtype", CoreAnnotations.NumericTypeAnnotation.class);
3. Bind patterns / strings for compiling patterns

   
       // Bind string for later compilation using: compile("/it/ /was/ $RELDAY");
       env.bind("$RELDAY", "/today|yesterday|tomorrow|tonight|tonite/");
       // Bind pre-compiled patter for later compilation using: compile("/it/ /was/ $RELDAY");
       env.bind("$RELDAY", TokenSequencePattern.compile(env, "/today|yesterday|tomorrow|tonight|tonite/"));
       

4. Bind custom node pattern functions (currently no arguments are supported)

   
       // Bind node pattern so we can do patterns like: compile("... temporal::IS_TIMEX_DATE ...");
       //   (TimexTypeMatchNodePattern is a NodePattern that implements some custom logic)
       env.bind("::IS_TIMEX_DATE", new TimexTypeMatchNodePattern(SUTime.TimexType.DATE));
      

Actions (partially implemented)

• pattern ==> action
• Supported action: &annotate( { ner="DATE" } )
• Not applied automatically, associated with a pattern.
• To apply, call pattern.getAction().apply(match, groupid)

Author:

Angel Chang

See Also:

TokenSequenceMatcher

Nested Class Summary

Nested classes/interfaces inherited from class edu.stanford.nlp.ling.tokensregex.SequencePattern

SequencePattern.AndPatternExpr, SequencePattern.BackRefPatternExpr, SequencePattern.GroupPatternExpr, SequencePattern.MultiNodePatternExpr, SequencePattern.NodePatternExpr, SequencePattern.NodesMatchChecker<T>, SequencePattern.OrPatternExpr, SequencePattern.Parser<T>, SequencePattern.PatternExpr, SequencePattern.RepeatPatternExpr, SequencePattern.SequenceEndPatternExpr, SequencePattern.SequencePatternExpr, SequencePattern.SequenceStartPatternExpr, SequencePattern.SpecialNodePatternExpr, SequencePattern.ValuePatternExpr

Field Summary

static TokenSequencePattern

ANY_NODE_PATTERN

Fields inherited from class edu.stanford.nlp.ling.tokensregex.SequencePattern

ANY_NODE_PATTERN_EXPR, MATCH_STATE, NODES_EQUAL_CHECKER, SEQ_BEGIN_PATTERN_EXPR, SEQ_END_PATTERN_EXPR

Constructor Summary

TokenSequencePattern(java.lang.String patternStr, SequencePattern.PatternExpr nodeSequencePattern)


TokenSequencePattern(java.lang.String patternStr, SequencePattern.PatternExpr nodeSequencePattern, SequenceMatchAction<CoreMap> action)


Method Summary

static TokenSequencePattern	compile(Env env, java.lang.String... strings) Compiles a sequence of regular expression a TokenSequencePattern using the specified environment
static TokenSequencePattern	compile(Env env, java.lang.String string) Compiles a regular expression over tokens into a TokenSequencePattern using the specified environment
static TokenSequencePattern	compile(SequencePattern.PatternExpr nodeSequencePattern)
static TokenSequencePattern	compile(java.lang.String... strings) Compiles a sequence of regular expression a TokenSequencePattern using the default environment
static TokenSequencePattern	compile(java.lang.String string) Compiles a regular expression over tokens into a TokenSequencePattern using the default environment
TokenSequenceMatcher	getMatcher(java.util.List<? extends CoreMap> tokens) Returns a TokenSequenceMatcher that can be used to match this pattern against the specified list of tokens
static MultiPatternMatcher<CoreMap>	getMultiPatternMatcher(java.util.Collection<TokenSequencePattern> patterns)
static MultiPatternMatcher<CoreMap>	getMultiPatternMatcher(TokenSequencePattern... patterns)
static Env	getNewEnv()
java.lang.String	toString()

Methods inherited from class edu.stanford.nlp.ling.tokensregex.SequencePattern

findNodePattern, getAction, getPatternExpr, getPriority, pattern, setAction, setPriority

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

ANY_NODE_PATTERN

public static final TokenSequencePattern ANY_NODE_PATTERN

Constructor Detail

TokenSequencePattern

public TokenSequencePattern(java.lang.String patternStr,
                            SequencePattern.PatternExpr nodeSequencePattern)

TokenSequencePattern

public TokenSequencePattern(java.lang.String patternStr,
                            SequencePattern.PatternExpr nodeSequencePattern,
                            SequenceMatchAction<CoreMap> action)

Method Detail

getNewEnv

public static Env getNewEnv()

compile

public static TokenSequencePattern compile(java.lang.String string)

Compiles a regular expression over tokens into a TokenSequencePattern using the default environment

Parameters:

string - Regular expression to be compiled

Returns:

Compiled TokenSequencePattern

compile

public static TokenSequencePattern compile(Env env,
                                           java.lang.String string)

Compiles a regular expression over tokens into a TokenSequencePattern using the specified environment

Parameters:

env - Environment to use

string - Regular expression to be compiled

Returns:

Compiled TokenSequencePattern

compile

public static TokenSequencePattern compile(java.lang.String... strings)

Compiles a sequence of regular expression a TokenSequencePattern using the default environment

Parameters:

strings - List of regular expression to be compiled

Returns:

Compiled TokenSequencePattern

compile

public static TokenSequencePattern compile(Env env,
                                           java.lang.String... strings)

Compiles a sequence of regular expression a TokenSequencePattern using the specified environment

Parameters:

env - Environment to use

strings - List of regular expression to be compiled

Returns:

Compiled TokenSequencePattern

compile

public static TokenSequencePattern compile(SequencePattern.PatternExpr nodeSequencePattern)

getMatcher

public TokenSequenceMatcher getMatcher(java.util.List<? extends CoreMap> tokens)

Returns a TokenSequenceMatcher that can be used to match this pattern against the specified list of tokens

Overrides:

getMatcher in class SequencePattern<CoreMap>

Parameters:

tokens - List of tokens to match against

Returns:

TokenSequenceMatcher

toString

public java.lang.String toString()

Overrides:

toString in class SequencePattern<CoreMap>

getMultiPatternMatcher

public static MultiPatternMatcher<CoreMap> getMultiPatternMatcher(java.util.Collection<TokenSequencePattern> patterns)

getMultiPatternMatcher

public static MultiPatternMatcher<CoreMap> getMultiPatternMatcher(TokenSequencePattern... patterns)