Implementing a lexical analyzer in Compiler Design - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to define the function that starts the lexical analysis.

Compiler Design

def [1](source_code):

Drag options to blanks, or click blank then click option'

Alexical_analyzer

Btokenize

Cparse

Dcompile

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'parse' which is for syntax analysis, not lexical analysis.

Using 'compile' which is for the whole compilation process.

The function that begins the lexical analysis is commonly called lexical_analyzer or similar, as it processes the source code into tokens.

2fill in blank

medium

Complete the code to read the next character from the source code string.

Compiler Design

current_char = source_code[[1]]

Drag options to blanks, or click blank then click option'

A0

Bindex

Clength

Dtoken

Attempts:

3 left

💡 Hint

Common Mistakes

Using '0' which always reads the first character, not the current one.

Using 'token' which is unrelated to string indexing.

The variable index is typically used to track the current position in the source code string to read characters one by one.

3fill in blank

hard

Fix the error in the condition to check if the current character is a digit.

Compiler Design

if current_char [1] '0' and current_char [2] '9':

Drag options to blanks, or click blank then click option'

A>=

B<=

C==

D!=

Attempts:

3 left

💡 Hint

Common Mistakes

Using '==' which only checks equality to one character.

Using '!=' which checks inequality, not range.

To check if a character is a digit, it must be greater than or equal to '0' and less than or equal to '9'.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that maps tokens to their types for all tokens longer than 2 characters.

Compiler Design

token_types = {token: [1] for token in tokens if len(token) [2] 2}

Drag options to blanks, or click blank then click option'

A'identifier'

B>

C==

D<

Attempts:

3 left

💡 Hint

Common Mistakes

Using '==' instead of '>' which limits tokens to length exactly 2.

Using '<' which selects tokens shorter than 2 characters.

The dictionary maps each token to the string 'identifier' if its length is greater than 2.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that maps uppercase tokens to their lengths if length is at least 3.

Compiler Design

result = { [1]: [2] for [3] in tokens if len([3]) >= 3 }

Drag options to blanks, or click blank then click option'

Atoken.upper()

Blen(token)

Ctoken

Dword

Attempts:

3 left

💡 Hint

Common Mistakes

Using different variable names inconsistently.

Not converting the token to uppercase for the key.

The comprehension maps each token converted to uppercase (token.upper()) to its length (len(token)) for tokens named token with length at least 3.

Practice

1. What is the main job of a lexical analyzer in compiler design?

easy

A. Break input text into meaningful tokens

B. Execute the program code

C. Optimize the compiled code

D. Generate machine code

Solution

1. Step 1: Understand the role of lexical analyzer

   A lexical analyzer reads the input text and divides it into tokens like keywords, identifiers, and symbols.

2. Step 2: Differentiate from other compiler phases

   It does not execute code, optimize, or generate machine code; those are later phases.

3. Final Answer:

   Break input text into meaningful tokens -> Option A

4. Quick Check:

   Lexical analyzer = tokenization [OK]

Hint: Lexical analyzer splits text into tokens [OK]

Common Mistakes:

• Confusing lexical analysis with code execution
• Thinking it optimizes or generates machine code
• Assuming it handles syntax errors

2. Which of the following is a correct way to define a token pattern in a lexical analyzer?

easy

A. token = (123)+

B. token = [a-zA-Z_][a-zA-Z0-9_]*

C. token = {print}

D. token = 123abc

Solution

1. Step 1: Recognize token pattern syntax

   Token patterns use regular expressions; [a-zA-Z_][a-zA-Z0-9_]* matches identifiers starting with letter or underscore.

2. Step 2: Check other options for correctness

   token = (123)+ uses invalid syntax for token pattern; C uses braces incorrectly; D is not a valid pattern.

3. Final Answer:

   token = [a-zA-Z_][a-zA-Z0-9_]* -> Option B

4. Quick Check:

   Valid regex pattern = token = [a-zA-Z_][a-zA-Z0-9_]* [OK]

Hint: Token patterns use regex-like syntax starting with letters or underscore [OK]

Common Mistakes:

• Using invalid characters in token patterns
• Confusing literal strings with patterns
• Ignoring start character rules

3. Given the input string int x = 10;, what tokens will a lexical analyzer produce?

medium

A. ["int", "x=10;"]

B. ["int x", "= 10;"]

C. ["int", "x", "=", "10", ";"]

D. ["int", "x", "=", "1", "0", ";"]

Solution

1. Step 1: Identify tokens in the input

   The lexical analyzer splits by language rules: keyword 'int', identifier 'x', operator '=', number '10', and symbol ';'.

2. Step 2: Compare with options

   ["int", "x", "=", "10", ";"] correctly lists all tokens separately. Others combine tokens incorrectly or split numbers wrongly.

3. Final Answer:

   ["int", "x", "=", "10", ";"] -> Option C

4. Quick Check:

   Tokens = separate meaningful units [OK]

Hint: Tokens separate keywords, identifiers, operators, numbers, symbols [OK]

Common Mistakes:

• Combining multiple tokens into one
• Splitting numbers into digits
• Ignoring punctuation as tokens

4. Identify the error in this lexical analyzer rule: NUMBER = [0-9]+(.[0-9]+)?

medium

A. Plus (+) should be replaced with star (*)

B. Parentheses should be removed

C. Square brackets are not allowed

D. Dot (.) should be escaped as \.

Solution

1. Step 1: Analyze the regex pattern

   The dot (.) in regex means any character; to match a literal dot, it must be escaped as \.

2. Step 2: Check other parts of the pattern

   Plus (+) is correct for one or more digits; square brackets and parentheses are valid here.

3. Final Answer:

   Dot (.) should be escaped as \. -> Option D

4. Quick Check:

   Escape dot for literal period [OK]

Hint: Escape dot in regex to match literal '.' [OK]

Common Mistakes:

• Not escaping dot causing wrong matches
• Changing quantifiers unnecessarily
• Removing valid grouping parentheses

5. How can a lexical analyzer handle comments in source code effectively?

hard

A. By recognizing comment patterns and skipping them

B. By converting comments into tokens

C. By treating comments as syntax errors

D. By storing comments as identifiers

Solution

1. Step 1: Understand comment handling

   Comments are ignored by the compiler; the lexical analyzer must detect comment start and end patterns and skip their content.

2. Step 2: Evaluate other options

   Converting comments into tokens or treating them as errors is incorrect; storing as identifiers is wrong.

3. Final Answer:

   By recognizing comment patterns and skipping them -> Option A

4. Quick Check:

   Skip comments during tokenization [OK]

Hint: Detect and skip comments, don't tokenize them [OK]

Common Mistakes:

• Tokenizing comments as code
• Raising errors on comments
• Confusing comments with identifiers