Lucene - TokenStream


Advertisements


TokenStream is an output of the analysis process and it comprises of a series of tokens. It is an abstract class.

Class Declaration

Following is the declaration for the org.apache.lucene.analysis.TokenStream class −

public abstract class TokenStream
   extends AttributeSource
      implements Closeable

Class Constructors

The following table shows the different class constructors −

S.No. Constructor & Description
1

protected TokenStream()

A TokenStream that uses the default attribute factory.

2

protected TokenStream(AttributeSource.AttributeFactory factory)

A TokenStream that uses the supplied AttributeFactory for creating new Attribute instances.

3

protected TokenStream(AttributeSource input)

A TokenStream that uses the same attributes as the supplied one.

Class Methods

The following table shows the different class methods −

S.No. Method & Description
1

void close()

Releases resources associated with this stream.

2

void end()

This method is called by the consumer after the last token has been consumed, after incrementToken() returned false (using the new TokenStream API).

3

abstract boolean incrementToken()

Consumers (i.e., IndexWriter) use this method to advance the stream to the next token.

4

void reset()

Resets this stream to the beginning.

Methods Inherited

This class inherits methods from the following classes −

  • org.apache.lucene.util.AttributeSource
  • java.lang.Object

lucene_analysis.htm

Advertisements