Use of StreamTokenizer

StreamTokenizer significance

The StreamTokenizer class is used to break any InputStream into a sequence of “tokens,” which are bits of text delimited by whatever we will choose. A stream tokenizer takes an input stream and parses it into tokens and it is allowing the tokens to be read one at a time.


//This is the sample program for counting words and numbers in a file available in file system.

package com.corejava.gaurav.examples;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.StreamTokenizer;


public class countWordsAndNumbersUsingStreamToenizer {
   
    /**
     * Example method for using the StreamTokenizer class
     */
    public void countWordsAndNumbers(String filename) {
       
        StreamTokenizer sTokenizer = null;
        int wordCount = 0, numberCount = 0;
       
        try {
           
            sTokenizer = new StreamTokenizer(new FileReader(filename));
           
            while (sTokenizer.nextToken() != StreamTokenizer.TT_EOF) {
               
                if (sTokenizer.ttype == StreamTokenizer.TT_WORD)
                    wordCount++;
                else if (sTokenizer.ttype == StreamTokenizer.TT_NUMBER)
                    numberCount++;
            }
           
            System.out.println("Number of words in file: " + wordCount);
            System.out.println("Number of numbers in file: " + numberCount);
           
        } catch (FileNotFoundException ex) {
            ex.printStackTrace();
        } catch (IOException ex) {
            ex.printStackTrace();
        }
    }
   
    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        new countWordsAndNumbersUsingStreamToenizer().countWordsAndNumbers("D://welcome.txt");
    }
}

Note: - Below is the contents of welcome.txt file

India is second biggest Country in Asia
234 hello 123 Welcome
7612 242542 4525 india
is great 145.( Sample text).