Java - StreamTokenizer whitespaceChars(int low, int hi) method

---

---

Description

The Java StreamTokenizer whitespaceChars(int low, int hi) method specifies that all characters c in the range low <= c <= high are white space characters. White space characters serve only to separate tokens in the input stream. Any other attribute settings for the characters in the specified range are cleared.

Declaration

Following is the declaration for java.io.StreamTokenizer.whitespaceChars(int low, int hi) method.

public void whitespaceChars(int low, int hi)

Parameters

• low − The low end of the range.

• high − The high end of the range.

Return Value

This method does not return a value.

Exception

NA

Example - Usage of StreamTokenizer whitespaceChars(int low, int hi) method

The following example shows the usage of StreamTokenizer whitespaceChars(int low, int hi) method.

StreamTokenizerDemo.java

package com.tutorialspoint;

import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Reader;
import java.io.StreamTokenizer;

public class StreamTokenizerDemo {
   public static void main(String[] args) {
      String text = "Hello. This is a text \n that will be split "
         + "into tokens. 1 + 1 = 2";
         
      try {
         // create a new file with an ObjectOutputStream
         FileOutputStream out = new FileOutputStream("test.txt");
         ObjectOutputStream oout = new ObjectOutputStream(out);

         // write something in the file
         oout.writeUTF(text);
         oout.flush();

         // create an ObjectInputStream for the file we created before
         ObjectInputStream ois = new ObjectInputStream(new FileInputStream("test.txt"));

         // create a new tokenizer
         Reader r = new BufferedReader(new InputStreamReader(ois));
         StreamTokenizer st = new StreamTokenizer(r);

         // set letters o- t as white space chars
         st.whitespaceChars('o', 't');

         // print the stream tokens
         boolean eof = false;
         
         do {
            int token = st.nextToken();

            switch (token) {
               case StreamTokenizer.TT_EOF:
                  System.out.println("End of File encountered.");
                  eof = true;
                  break;
                  
               case StreamTokenizer.TT_EOL:
                  System.out.println("End of Line encountered.");
                  break;
                  
               case StreamTokenizer.TT_WORD:
                  System.out.println("Word: " + st.sval);
                  break;
                  
               case StreamTokenizer.TT_NUMBER:
                  System.out.println("Number: " + st.nval);
                  break;
                  
               default:
                  System.out.println((char) token + " encountered.");
                  
                  if (token == '!') {
                     eof = true;
                  }
            }
         } while (!eof);

      } catch (Exception ex) {
         ex.printStackTrace();
      }
   }
}

Output

Let us compile and run the above program, this will produce the following result −

Word: AHell
Number: 0.0
Word: Thi
Word: i
Word: a
Word: ex
Word: ha
Word: will
Word: be
Word: li
Word: in
Word: ken
Number: 0.0
Number: 1.0
+ encountered.
Number: 1.0
= encountered.
Number: 2.0
End of File encountered.

Example - Treat comma (,) and space as whitespace

The following example shows the usage of StreamTokenizer whitespaceChars(int low, int hi) method.

StreamTokenizerDemo.java

package com.tutorialspoint;

import java.io.IOException;
import java.io.Reader;
import java.io.StreamTokenizer;
import java.io.StringReader;

public class StreamTokenizerDemo {
   public static void main(String[] args) throws IOException {
      String input = "Java,Python C++";

      Reader reader = new StringReader(input);
      StreamTokenizer tokenizer = new StreamTokenizer(reader);

      // Treat space and comma as whitespace
      tokenizer.whitespaceChars(32, 32);  // space
      tokenizer.whitespaceChars(',', ','); // comma
      tokenizer.wordChars('a', 'z');
      tokenizer.wordChars('A', 'Z');
      tokenizer.wordChars('+', '+'); // to include '+' in C++

      System.out.println("Tokens:");
      while (tokenizer.nextToken() != StreamTokenizer.TT_EOF) {
         if (tokenizer.ttype == StreamTokenizer.TT_WORD) {
            System.out.println("Word: " + tokenizer.sval);
         }
      }
   }
}

Output

Let us compile and run the above program, this will produce the following result−

Tokens:
Word: Java
Word: Python
Word: C++

Explanation

• Both comma and space are whitespace, so tokens are split by them.

• They don't appear in the output.

Example - Treat digits 0-9 as whitespace (stripping them out)

The following example shows the usage of StreamTokenizer whitespaceChars(int low, int hi) method.

StreamTokenizerDemo.java

package com.tutorialspoint;

import java.io.IOException;
import java.io.Reader;
import java.io.StreamTokenizer;
import java.io.StringReader;

public class StreamTokenizerDemo {
   public static void main(String[] args) throws IOException {
      String input = "a1b2c3";

      Reader reader = new StringReader(input);
      StreamTokenizer tokenizer = new StreamTokenizer(reader);

      tokenizer.whitespaceChars('0', '9'); // treat all digits as whitespace
      tokenizer.wordChars('a', 'z');
      tokenizer.wordChars('A', 'Z');

      System.out.println("Tokens:");
      while (tokenizer.nextToken() != StreamTokenizer.TT_EOF) {
         if (tokenizer.ttype == StreamTokenizer.TT_WORD) {
            System.out.println("Word: " + tokenizer.sval);
         }
      }
   }
}

Output

Let us compile and run the above program, this will produce the following result−

Tokens:
Word: a
Word: b
Word: c

Explanation

• The digits 1, 2, and 3 are treated as whitespace.

• They are used to split the words but do not appear as tokens.