TokenStream is an output of the analysis process and it comprises of a series of tokens. It is an abstract class.
Following is the declaration for the org.apache.lucene.analysis.TokenStream class −
public abstract class TokenStream extends AttributeSource implements Closeable
The following table shows the different class constructors −
|S.No.||Constructor & Description|
A TokenStream that uses the default attribute factory.
protected TokenStream(AttributeSource.AttributeFactory factory)
A TokenStream that uses the supplied AttributeFactory for creating new Attribute instances.
protected TokenStream(AttributeSource input)
A TokenStream that uses the same attributes as the supplied one.
The following table shows the different class methods −
|S.No.||Method & Description|
Releases resources associated with this stream.
This method is called by the consumer after the last token has been consumed, after incrementToken() returned false (using the new TokenStream API).
abstract boolean incrementToken()
Consumers (i.e., IndexWriter) use this method to advance the stream to the next token.
Resets this stream to the beginning.
This class inherits methods from the following classes −