Overview | Package | Class | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## edu.stanford.nlp.trees
Class PennTreebankTokenizer

    java.lang.Object
      edu.stanford.nlp.process.AbstractTokenizer<String>
          edu.stanford.nlp.process.TokenizerAdapter
              edu.stanford.nlp.trees.PennTreebankTokenizer

All Implemented Interfaces:
  Tokenizer<String>, Iterator<String>

    public class PennTreebankTokenizer

extends TokenizerAdapter

Builds a tokenizer for English PennTreebank (release 2) trees. This is currently internally implemented via a java.io.StreamTokenizer.

Author:
  Christopher Manning, Roger Levy

Field Summary

Fields inherited from class edu.stanford.nlp.process.TokenizerAdapter
`eolString, st`

Fields inherited from class edu.stanford.nlp.process.AbstractTokenizer
`nextToken`

Constructor Summary
`PennTreebankTokenizer(Reader r)`

Method Summary

Methods inherited from class edu.stanford.nlp.process.TokenizerAdapter
`getNext, isEol, setEolString`

Methods inherited from class edu.stanford.nlp.process.AbstractTokenizer
`hasNext, next, peek, remove, tokenize`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Constructor Detail

### PennTreebankTokenizer

    public PennTreebankTokenizer(Reader r)

Overview | Package | Class | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Stanford NLP Group