SAS: Program compilation - word scanner

The word scanner identifies (or builds) tokens during the compilation process.

In order to build those tokens, the word scanner reads one character after another off the input stack.

After having created one token, the token is either passed to the word queue or to the the multi component macro processor.