Lucene - TokenStream


Advertisements


Introduction

TokenStream is an output of analysis process and it comprises of series of tokens. It is an abstract class.

Class declaration

Following is the declaration for org.apache.lucene.analysis.TokenStream class:

public abstract class TokenStream
   extends AttributeSource
      implements Closeable

Class constructors

S.N.Constructor & Description
1

protected TokenStream()

A TokenStream using the default attribute factory.

2

protected TokenStream(AttributeSource.AttributeFactory factory)

A TokenStream using the supplied AttributeFactory for creating new Attribute instances.

3

protected TokenStream(AttributeSource input)

A TokenStream that uses the same attributes as the supplied one.

Class methods

S.N.Method & Description
1

void close()

Releases resources associated with this stream.

2

void end()

This method is called by the consumer after the last token has been consumed, after incrementToken() returned false (using the new TokenStream API).

3

abstract boolean incrementToken()

Consumers (i.e., IndexWriter) use this method to advance the stream to the next token.

4

void reset()

Resets this stream to the beginning.

Methods inherited

This class inherits methods from the following classes:

  • org.apache.lucene.util.AttributeSource

  • java.lang.Object


lucene_analysis.htm

Advertisements