Document loading and parsing in Prompt Engineering / GenAI - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is document loading in the context of AI?

Document loading is the process of bringing text or data files into a program so they can be read and used for AI tasks like analysis or training.

beginner

Why is parsing important after loading a document?

Parsing breaks down the loaded document into smaller parts like sentences or words, making it easier for AI models to understand and work with the content.

beginner

Name two common formats for documents that AI systems often load and parse.

Common document formats include plain text (.txt) and PDF (.pdf). AI systems use special tools to read and parse these formats.

beginner

What is a real-life example of document loading and parsing?

When you upload a resume to a job site, the system loads your document and parses it to find your skills and experience automatically.

intermediate

What could happen if a document is loaded but not parsed correctly?

If parsing fails, the AI might misunderstand the content, leading to wrong answers or poor analysis because it can't read the document properly.

What is the first step in using a document for AI analysis?

A Parsing the document

B Training the AI model

C Making predictions

D Loading the document into the system

Before parsing or analysis, the document must be loaded into the system.

Parsing a document helps AI by:

A Breaking it into understandable parts

B Deleting unnecessary data

C Changing the document format

D Encrypting the content

Parsing breaks the document into smaller parts like words or sentences for easier understanding.

Which file format might require special tools to parse?

A .csv

B .txt

C .pdf

D .json

PDF files often need special libraries to extract and parse text.

If a document is not parsed correctly, what is a likely outcome?

A The AI will work faster

B The AI might misunderstand the content

C The document will be deleted

D The AI will ignore the document

Incorrect parsing can cause the AI to misinterpret the document's content.

Which step comes directly after loading a document?

A Parsing the document

B Making predictions

C Training the model

D Saving the document

After loading, the document is parsed to prepare it for AI processing.

Explain the process and importance of document loading and parsing in AI.

Think about how AI reads and prepares text data before analysis.

Describe a real-world example where document loading and parsing helps AI provide useful results.

Consider everyday tasks like resume screening or email sorting.

Practice

1. What is the main purpose of document loading in AI projects?

easy

A. To clean the data by removing errors

B. To train the AI model with labeled data

C. To visualize the results of the AI model

D. To read text files so the computer can access their content

Solution

1. Step 1: Understand document loading

   Document loading means reading text files so the computer can access the content inside.

2. Step 2: Differentiate from other tasks

   Training models, visualization, and cleaning are different steps after loading the document.

3. Final Answer:

   To read text files so the computer can access their content -> Option D

4. Quick Check:

   Document loading = reading files [OK]

Hint: Loading means reading files into the computer [OK]

Common Mistakes:

• Confusing loading with training the model
• Thinking loading cleans the data
• Mixing loading with visualization

2. Which Python code snippet correctly loads a text file named data.txt into a string variable?

easy

A. with open('data.txt', 'x') as file:
text = file.read()

B. file = open('data.txt', 'w')
text = file.read()

C. with open('data.txt', 'r') as file:
text = file.read()

D. text = open('data.txt').write()

Solution

1. Step 1: Check file mode for reading

   Mode 'r' opens the file for reading, which is needed to load text.

2. Step 2: Use context manager and read method

   Using with open(...) ensures safe file handling, and file.read() reads all content.

3. Final Answer:

   with open('data.txt', 'r') as file: text = file.read() -> Option C

4. Quick Check:

   Open with 'r' and read() = correct loading [OK]

Hint: Use 'r' mode and read() to load text files [OK]

Common Mistakes:

• Using 'w' mode which is for writing, not reading
• Calling write() instead of read()
• Using 'x' mode which is for creating new files

3. What will be the output of this Python code that parses a loaded text?

text = "Hello world! Welcome to AI."
words = text.split()
print(words)

medium

A. ['Hello', 'world', 'Welcome', 'to', 'AI']

B. ['Hello', 'world!', 'Welcome', 'to', 'AI.']

C. ['Hello world! Welcome to AI.']

D. ['H', 'e', 'l', 'l', 'o']

Solution

1. Step 1: Understand split() method

   The split() method splits the string by spaces into a list of words, keeping punctuation attached.

2. Step 2: Apply split() to the text

   Splitting "Hello world! Welcome to AI." results in ['Hello', 'world!', 'Welcome', 'to', 'AI.'] including punctuation.

3. Final Answer:

   ['Hello', 'world!', 'Welcome', 'to', 'AI.'] -> Option B

4. Quick Check:

   split() by space keeps punctuation attached [OK]

Hint: split() breaks text by spaces, punctuation stays [OK]

Common Mistakes:

• Expecting punctuation to be removed automatically
• Thinking split() returns a single string list
• Confusing split() with list(text) which splits characters

4. Identify the error in this code that tries to parse a document into sentences:

text = "AI is fun. Let's learn it."
sentences = text.split('. ')
print(sentences)

medium

A. The split delimiter '. ' misses the last sentence ending

B. The code should use splitlines() instead of split()

C. The print statement is missing parentheses

D. The variable name 'sentences' is invalid

Solution

1. Step 1: Analyze split delimiter usage

   Splitting by '. ' splits sentences but leaves the last sentence without a trailing '. ' unseparated.

2. Step 2: Understand effect on last sentence

   The last sentence "Let's learn it." remains attached with the period, causing inconsistent splitting.

3. Final Answer:

   The split delimiter '. ' misses the last sentence ending -> Option A

4. Quick Check:

   Splitting by '. ' misses last sentence split [OK]

Hint: Splitting by '. ' misses last sentence if no trailing space [OK]

Common Mistakes:

• Thinking splitlines() splits sentences
• Forgetting print() needs parentheses in Python 3
• Assuming variable names cause errors

5. You have a text file with multiple paragraphs separated by blank lines. Which approach best loads and parses it into a list of paragraphs for AI processing?

hard

A. Read the file, split text by double newlines '\n\n', then strip whitespace from each paragraph

B. Read the file line by line and treat each line as a paragraph

C. Use split() to split by single spaces to get paragraphs

D. Load the file and convert all text to uppercase without splitting

Solution

1. Step 1: Understand paragraph separation

   Paragraphs are separated by blank lines, which means two newline characters '\n\n'.

2. Step 2: Parse paragraphs correctly

   Splitting by '\n\n' divides text into paragraphs; stripping whitespace cleans each paragraph.

3. Final Answer:

   Read the file, split text by double newlines '\n\n', then strip whitespace from each paragraph -> Option A

4. Quick Check:

   Split by '\n\n' for paragraphs [OK]

Hint: Paragraphs split by double newlines '\n\n' [OK]

Common Mistakes:

• Splitting by single spaces splits words, not paragraphs
• Treating each line as a paragraph loses multi-line paragraphs
• Ignoring whitespace cleanup after splitting