Lucene - StopAnalyzer Analyzer

StopAnalyzer works similar to SimpleAnalyzer and remove the common words like 'a', 'an', 'the', etc.

Class Declaration

Following is the declaration for org.apache.lucene.analysis.core.StopAnalyzer class −

public final class StopAnalyzer
   extends Analyzer

Methods Inherited

This class inherits methods from the following classes −

• org.apache.lucene.analysis.StopwordAnalyzerBase
• org.apache.lucene.analysis.Analyzer
• java.lang.Object

Usage of StopAnalyzer

private void displayTokenUsingStopAnalyzer() throws IOException {
   String text = "The Lucene is a simple yet powerful java based search library.";

   Set<String> stopWords = new HashSet<>();
   stopWords.add("a");
   stopWords.add("an");
   stopWords.add("the");

   Analyzer analyzer = new StopAnalyzer(CharArraySet.copy(stopWords));
   TokenStream tokenStream = analyzer.tokenStream(
   LuceneConstants.CONTENTS, new StringReader(text));
   CharTermAttribute term = tokenStream.addAttribute(CharTermAttribute.class);
   tokenStream.reset();
   while(tokenStream.incrementToken()) {
      System.out.print("[" + term.toString() + "] ");
   }
   analyzer.close();
}

Example Application

To test search using StopAnalyzer, let us create a test Lucene application.

LuceneConstants.java

This class is used to provide various constants to be used across the sample application.

package com.tutorialspoint.lucene;

public class LuceneConstants {
   public static final String CONTENTS = "contents";
   public static final String FILE_NAME = "filename";
   public static final String FILE_PATH = "filepath";
   public static final int MAX_SEARCH = 10;
}

LuceneTester.java

This class is used to test the searching capability of the Lucene library.

package com.tutorialspoint.lucene;

import java.io.IOException;
import java.io.StringReader;
import java.util.HashSet;
import java.util.Set;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.CharArraySet;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.core.StopAnalyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;

public class LuceneTester {
	
   public static void main(String[] args) {
      LuceneTester tester;

      tester = new LuceneTester();
   
      try {
         tester.displayTokenUsingStopAnalyzer();
      } catch (IOException e) {
         e.printStackTrace();
      }
   }

   private void displayTokenUsingStopAnalyzer() throws IOException {
      String text 
         = "The Lucene is a simple yet powerful java based search library.";
      
      Set<String> stopWords = new HashSet<>();
      stopWords.add("a");
      stopWords.add("an");
      stopWords.add("the");
      
      Analyzer analyzer = new StopAnalyzer(CharArraySet.copy(stopWords));
      TokenStream tokenStream = analyzer.tokenStream(
         LuceneConstants.CONTENTS, new StringReader(text));
      CharTermAttribute term = tokenStream.addAttribute(CharTermAttribute.class);
      tokenStream.reset();
      while(tokenStream.incrementToken()) {
         System.out.print("[" + term.toString() + "] ");
      }
      analyzer.close();
   }
}

Running the Program

Once you are done with the creation of the source, the raw data, the data directory, the index directory and the indexes, you can proceed by compiling and running your program. To do this, keep the LuceneTester.Java file tab active and use either the Run option available in the Eclipse IDE or use Ctrl + F11 to compile and run your LuceneTester application. If your application runs successfully, it will print the following message in Eclipse IDE's console −

Output

[lucene] [is] [simple] [yet] [powerful] [java] [based] [search] [library]