Document loaders in Prompt Engineering / GenAI - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a document loader in machine learning?

A document loader is a tool or code that reads and imports documents (like text files, PDFs, or web pages) into a program so the data can be used for training or analysis.

beginner

Why do we need document loaders before training AI models?

Because AI models need data in a clean, organized format. Document loaders help convert raw documents into structured data that models can understand and learn from.

beginner

Name two common types of documents that document loaders handle.

Text files (like .txt) and PDFs are two common document types that loaders can read and process.

intermediate

How does a document loader handle different file formats?

It uses specific methods or libraries designed for each format to extract the text or data correctly, for example, using PDF parsers for PDFs and simple reading for text files.

intermediate

What is one challenge document loaders face when processing scanned documents?

Scanned documents are images, so loaders need Optical Character Recognition (OCR) to convert images of text into actual text data before processing.

What is the main purpose of a document loader?

A To read and import documents into a program

B To train machine learning models directly

C To create new documents automatically

D To delete unwanted files

Document loaders help bring documents into a program so the data can be used for AI tasks.

Which file type usually requires special parsing when using a document loader?

A Plain text (.txt)

B CSV files

C JSON files

D PDF files

PDF files have complex formatting and need special libraries to extract text properly.

What technology helps document loaders read text from scanned images?

A Speech Recognition

B Natural Language Processing

C Optical Character Recognition (OCR)

D Image Compression

OCR converts images of text into actual text data.

Which of these is NOT a function of a document loader?

A Training the AI model

B Extracting text from documents

C Cleaning and organizing data

D Handling multiple file formats

Training AI models is done after loading and preparing data, not by the loader itself.

Why is it important for document loaders to handle different formats?

A To encrypt the data

B Because data comes in many forms and formats

C To reduce file size

D To make documents look nicer

Data can be in text files, PDFs, or web pages, so loaders must handle all to prepare data well.

Explain what a document loader does and why it is important in AI projects.

Think about how raw documents become usable data for AI.

Describe a challenge document loaders face with scanned documents and how it is solved.

Consider how text in pictures becomes readable text.

Practice

1. What is the main purpose of a document loader in AI applications?

easy

A. To visualize data in charts and graphs

B. To train AI models directly from raw data

C. To read files and convert their content into a format machines can understand

D. To compress files for storage

Solution

1. Step 1: Understand the role of document loaders

   Document loaders are designed to read files and extract their content in a way that machines can process.

2. Step 2: Differentiate from other tasks

   Training models or visualizing data are separate steps after loading the data.

3. Final Answer:

   To read files and convert their content into a format machines can understand -> Option C

4. Quick Check:

   Document loader = read and convert files [OK]

Hint: Remember: loaders prepare data, not train or visualize [OK]

Common Mistakes:

• Confusing loading with training
• Thinking loaders compress files
• Assuming loaders create visualizations

2. Which of the following is the correct way to load a PDF file using a document loader in Python?

easy

A. loader = ImageLoader('file.pdf')

B. loader = PDFLoader('file.pdf')

C. loader = CSVLoader('file.pdf')

D. loader = TextLoader('file.pdf')

Solution

1. Step 1: Identify the correct loader for PDF files

   PDFLoader is designed specifically to read PDF documents.

2. Step 2: Check other loaders' purposes

   TextLoader is for plain text files, CSVLoader for CSV files, and ImageLoader for images, so they are incorrect for PDFs.

3. Final Answer:

   loader = PDFLoader('file.pdf') -> Option B

4. Quick Check:

   PDF file uses PDFLoader [OK]

Hint: Match loader type to file type exactly [OK]

Common Mistakes:

• Using TextLoader for PDFs
• Confusing CSVLoader with PDFLoader
• Trying to load PDFs as images

3. Given the following Python code snippet, what will be the output type of documents after loading a text file?

from langchain.document_loaders import TextLoader
loader = TextLoader('sample.txt')
documents = loader.load()

medium

A. An integer representing file size

B. A single string with all text combined

C. A dictionary with file metadata

D. A list of Document objects containing the text content

Solution

1. Step 1: Understand what TextLoader.load() returns

   The load() method returns a list of Document objects, each holding part or all of the file's text content.

2. Step 2: Eliminate other options

   It does not return a single string, dictionary, or integer.

3. Final Answer:

   A list of Document objects containing the text content -> Option D

4. Quick Check:

   TextLoader.load() returns list of Documents [OK]

Hint: Loaders return lists of Documents, not raw strings [OK]

Common Mistakes:

• Expecting a single string instead of list
• Thinking output is metadata dictionary
• Confusing output with file size

4. Identify the error in this code snippet for loading a PDF file:

from langchain.document_loaders import PDFLoader
loader = PDFLoader('document.txt')
docs = loader.load()

medium

A. The file extension does not match the loader type

B. Missing parentheses in load method

C. Incorrect import statement for PDFLoader

D. The variable name 'docs' is invalid

Solution

1. Step 1: Check file name and loader compatibility

   PDFLoader expects a PDF file, but the file given is 'document.txt', a text file.

2. Step 2: Verify other code parts

   Parentheses are correct, import is correct, and variable name is valid.

3. Final Answer:

   The file extension does not match the loader type -> Option A

4. Quick Check:

   Loader and file type must match [OK]

Hint: Match file extension to loader type to avoid errors [OK]

Common Mistakes:

• Ignoring file extension mismatch
• Thinking variable names cause errors
• Assuming import is wrong without checking

5. You want to load multiple document types (PDF, TXT, CSV) for an AI model training pipeline. Which approach best handles this using document loaders?

hard

A. Use separate loaders for each file type and combine their outputs into one list

B. Use only TextLoader for all files regardless of type

C. Convert all files to images and use ImageLoader

D. Load only PDF files and ignore others

Solution

1. Step 1: Understand file type differences

   Different file types require different loaders to correctly extract content.

2. Step 2: Combine outputs for unified processing

   Using separate loaders and merging their outputs ensures all data is loaded properly for training.

3. Final Answer:

   Use separate loaders for each file type and combine their outputs into one list -> Option A

4. Quick Check:

   Different loaders + combine outputs = best practice [OK]

Hint: Use correct loader per file, then merge results [OK]

Common Mistakes:

• Using one loader for all file types
• Ignoring non-PDF files
• Converting files unnecessarily to images