StreamTokenizer in Java

Stream Tokenizer takes an input stream(any String with seperator) and parses it into tokens, allowing the tokens to be read once at a time. 

 

 

                import java.io.StreamTokenizer; 

               import java.io.*;

             class ParseString

                   { 

              public static void main(String[] aa)

                     { 

                char separatorr = '.'; 

                StreamTokenizer tokenizerr = new StreamTokenizer(new InputStreamReader(System.in)); 

                tokenizerr.resetSyntax(); 

             

               

                tokenizerr.wordChars('\u0000', (char)(separatorr - 1));     // Everything is a word character 

                tokenizerr.wordChars((char)(separatorr + 1), '\u00ff');      // except for the separator 

                tokenizerr.whitespaceChars('\n', '\n');      // Make end-of-line whitespace(and therefore a word delimiter) 

                tokenizerr.whitespaceChars(separatorr, separatorr); // Delimiter separates words 

                tokenizerr.eolIsSignificant(true);  // End-of-line to be reported as TT_EOL 

             

                int typ = 0;            // Stores the value returned by nextToken() 

                String tokenn = null; // Will store the tokens that we find 

                try

      while((typ = tokenizerr.nextToken()) != StreamTokenizer.TT_EOL)// loop until end of file is reached 

                           {  

                    if(typ == StreamTokenizer.TT_WORD) //next token is a string

                            {                             // Check for a word 

                      System.out.println("  "  + tokenizerr.sval.trim());// and output it 

                       }

                          else

                              { 

                                 System.out.println("noooSorry");         

                                                                             // We only expect words 

                         } 

                  } 

                } catch(IOException e) { 

                  e.printStackTrace(); 

                  System.exit(1); 

                } 

              } 

            } 

StreamTokenizer in Java

 Explanation...

  //This Stream defines four int constants: TT_EOF, TT_EOL, TT_NUMBER, and TT_WORD.   //Stream Tokenizer takes an input stream(any String with seperator) and parses it into tokens, allowing the tokens to be read once at a time.

  //This Stream defines four int constants: TT_EOF, TT_EOL, TT_NUMBER, and TT_WORD.