Tokenizing

• When you want to read in data separated by delimiters (e.g., spaces, colons) you want to read tokens.
• Very useful for parsing.

import java.io.*;
import java.util.*;
public class StringTokenizerExample
{
  public static void main(String args[])
  {
    String line = "abc|def?ghi";
    //Sets the delimeters to | and ?
    StringTokenizer st = new StringTokenizer(line, "|?");
    while (st.hasMoreTokens())
    {
      String s = st.nextToken();
      System.out.println("Token is " + s);
    }
  }
}
//Output:
//Token is abc
//Token is def
//Token is ghi