Question

The output of a lexical analyzer is which one of the following?

• A. A parse tree
• B. Intermediate code
• C. Machine code
• D. A stream of tokens

Hint:

Lexical analysis converts characters into tokens, while syntax analysis converts tokens into a parse structure.

Answer 1

The Correct Option is D

Step 1: Understanding the role of a lexical analyzer.
The lexical analyzer (also called scanner) is the first phase of a compiler. It reads the source program as a sequence of characters.
Step 2: Identifying its main task.
Its primary function is to group characters into meaningful units called tokens, such as identifiers, keywords, operators, and literals.
Step 3: Output produced by lexical analysis.
After scanning the input, the lexical analyzer outputs a sequence (stream) of tokens, which is then passed to the syntax analyzer (parser).
Step 4: Final conclusion.
Hence, the correct output of a lexical analyzer is a stream of tokens.

Answer 2

The lexical analyzer is the very first phase of compilation, it reads the raw source characters. Let's check what each option would mean as its output:

1. A parse tree: Building a parse tree requires understanding the grammatical structure of the code, matching statements, expressions, and nesting, which is the job of the syntax analyzer (parser), a phase that comes after lexical analysis and consumes its output.
2. Intermediate code: Generating intermediate code (like three-address code) requires the program's meaning to already be understood at the semantic level, this happens much later, after parsing and semantic analysis, not at the character-scanning stage.
3. Machine code: This is the final output of the entire compiler, produced only after code generation and optimization, far downstream of lexical analysis.
4. A stream of tokens: The lexical analyzer's job is exactly to group raw characters into meaningful units, keywords, identifiers, operators, literals, punctuation, and hand these off one at a time (or as a full sequence) to the next phase. This is its defined output.

Every other option belongs to a later compiler phase that depends on the lexical analyzer having already run.

Therefore, the correct answer is A stream of tokens.