Class Scanner

java.lang.Object
  extended by Scanner

public class Scanner
extends java.lang.Object

Scan an input stream looking for tokens defined for a simple language.


Field Summary
private  int idx
          Zero-based offset to the next unaccepted character in the srcLine.
private  CompilerIO io
          The CompilerIO object that is managing all of our IO.
private  SymbolTable keywords
          Symbol table 0 stores the identifiers that are to be recognized as keywords, along with their specific Token.xyz values.
private  java.lang.String srcLine
          the current line of the source file, null if we have read to end of file.
 
Constructor Summary
Scanner(CompilerIO io, SymbolTable reserved)
          Construct a new Scanner object.
 
Method Summary
private  void acceptCurrentCharacter()
          Accept the previously returned character, and advance the index to the next available character.
private  char getCurrentCharacter()
          Get the current character from the source file.
private  void invalidCharacter(char c)
          Report invalid character.
 Token nextToken()
          Starting with the current character, identify the next Token and return the appropriate Token object to the caller.
private  void skipWhitespace()
          Skip over whitespace characters.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

keywords

private SymbolTable keywords
Symbol table 0 stores the identifiers that are to be recognized as keywords, along with their specific Token.xyz values.


io

private CompilerIO io
The CompilerIO object that is managing all of our IO.


srcLine

private java.lang.String srcLine
the current line of the source file, null if we have read to end of file.


idx

private int idx
Zero-based offset to the next unaccepted character in the srcLine. Points to the character that will be returned by getCurrentCharacter. idx is incremented every time acceptCurrentCharacter is called.

Constructor Detail

Scanner

public Scanner(CompilerIO io,
               SymbolTable reserved)
Construct a new Scanner object. The keywords symbol table is initialized here.

Parameters:
io - the CompilerIO object to use in reading and writing files.
reserved - the SymbolTable manager. Table 0 is assumed to be the reserved word table and is initialized in this constructor.
Method Detail

nextToken

public Token nextToken()
Starting with the current character, identify the next Token and return the appropriate Token object to the caller. This method calls the private helper methods getCurrentCharacter and acceptCurrentCharacter repeatedly to inspect and advance past characters until it decides that it has found something worthy of being reported as a Token.

Returns:
the appropriate Token object

skipWhitespace

private void skipWhitespace()
Skip over whitespace characters.


invalidCharacter

private void invalidCharacter(char c)
Report invalid character.

Parameters:
c - the invalid character

getCurrentCharacter

private char getCurrentCharacter()
Get the current character from the source file. If we are at EOF, then srcLine is null and this method returns 0. The caller should check for srcLine == null before using the returned char. If we are not at EOF, but we are at the end of this line, then we return a newline character ('\n'). If none of the above are true, then we are pointing to a character in the current srcLine and so we return that char. increment idx. idx always points to the current character. If it is equal to the length of the srcLine, then it is trying to point to the (virtual) newline.

Returns:
the currently available character.

acceptCurrentCharacter

private void acceptCurrentCharacter()
Accept the previously returned character, and advance the index to the next available character. This method must be called every time the scanner decides that it will be able to use the current character in whatever the current lexeme is. This is the only way to advance the index through the srcLine.