Dataset from files in TensorFlow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of using tf.data.Dataset.from_tensor_slices() when working with files?

It creates a dataset by slicing a tensor (like a list of file paths), allowing TensorFlow to read and process each file one by one in a pipeline.

beginner

How does tf.data.TextLineDataset help when loading data from text files?

It reads lines from one or more text files and creates a dataset where each element is a line of text, useful for processing text data line-by-line.

intermediate

Why is it useful to use map() on a TensorFlow dataset created from files?

The map() function applies a transformation to each element (like decoding images or parsing text), making data ready for training or analysis.

beginner

What does batching do in a dataset pipeline created from files?

Batching groups multiple data samples into one batch, which speeds up training by processing many samples at once instead of one by one.

intermediate

How can you shuffle data when loading from files using TensorFlow datasets?

You use the shuffle(buffer_size) method on the dataset to randomly mix the order of data elements, helping models learn better by reducing bias.

Which TensorFlow function creates a dataset from a list of file paths?

A tf.data.TextLineDataset

B tf.data.Dataset.from_tensor_slices

C tf.io.read_file

D tf.data.Dataset.batch

tf.data.Dataset.from_tensor_slices takes a list (tensor) of file paths and creates a dataset that can be used to read each file.

What does tf.data.TextLineDataset do?

A Reads entire files as single elements

B Shuffles dataset elements

C Reads lines from text files as dataset elements

D Creates batches of images

It reads text files line-by-line, making each line an element in the dataset.

Why use map() on a dataset created from files?

A To apply a function to each data element

B To batch the data

C To shuffle the data

D To split the dataset

map() applies a function to each element, like decoding or preprocessing.

What is the benefit of batching data in TensorFlow datasets?

A It speeds up training by processing multiple samples at once

B It reduces the dataset size

C It shuffles the data

D It reads files faster

Batching groups samples so the model can train on many at once, improving speed.

How do you randomize the order of data samples in a TensorFlow dataset?

A Using <code>batch()</code>

B Using <code>repeat()</code>

C Using <code>map()</code>

D Using <code>shuffle()</code>

shuffle() mixes the order of dataset elements randomly.

Explain how to create a TensorFlow dataset from a list of image file paths and prepare it for training.

Think about reading files, processing each image, and organizing data for training.

Describe the role of the map() function in a TensorFlow dataset pipeline when loading data from files.

Consider how raw data becomes ready for the model.

Practice

1. What is the main purpose of using tf.data.Dataset.from_tensor_slices() with file paths in TensorFlow?

easy

A. To convert tensors into image files

B. To directly read image data from files into memory

C. To save datasets to disk as files

D. To create a dataset that holds file paths which can be read later

Solution

1. Step 1: Understand the function purpose

   tf.data.Dataset.from_tensor_slices() creates a dataset from a tensor, often a list of file paths, not the file contents themselves.

2. Step 2: Clarify dataset content

   The dataset holds file paths as strings, which can be mapped later to read actual file data.

3. Final Answer:

   To create a dataset that holds file paths which can be read later -> Option D

4. Quick Check:

   from_tensor_slices(file_paths) = dataset of paths [OK]

Hint: Remember: from_tensor_slices holds paths, not file data [OK]

Common Mistakes:

• Thinking it reads file contents immediately
• Confusing dataset creation with saving files
• Assuming it converts tensors to images

2. Which of the following is the correct way to create a dataset from a list of image file paths in TensorFlow?

easy

A. dataset = tf.data.Dataset.from_tensor_slices(image_paths)

B. dataset = tf.data.Dataset.read_files(image_paths)

C. dataset = tf.data.Dataset.load(image_paths)

D. dataset = tf.data.Dataset.create(image_paths)

Solution

1. Step 1: Recall correct TensorFlow method

   The method to create a dataset from a list of tensors (like file paths) is from_tensor_slices().

2. Step 2: Verify options

   Methods like tf.data.Dataset.load(), tf.data.Dataset.read_files(), and tf.data.Dataset.create() are not valid TensorFlow dataset creation methods.

3. Final Answer:

   dataset = tf.data.Dataset.from_tensor_slices(image_paths) -> Option A

4. Quick Check:

   Correct method is from_tensor_slices [OK]

Hint: Use from_tensor_slices for lists of file paths [OK]

Common Mistakes:

• Using non-existent methods like read_files or load
• Confusing dataset creation with file reading
• Misspelling method names

3. Given the code below, what will be the output when iterating over the dataset?

import tensorflow as tf
image_paths = ["img1.jpg", "img2.jpg"]
dataset = tf.data.Dataset.from_tensor_slices(image_paths)
for item in dataset:
    print(item.numpy().decode())

medium

A. Error: decode() not found

B. [b'img1.jpg', b'img2.jpg']

C. img1.jpg\nimg2.jpg

D. Tensor objects printed

Solution

1. Step 1: Understand dataset content

   The dataset contains string tensors of file paths: b'img1.jpg', b'img2.jpg'.

2. Step 2: Decode bytes to string

   Calling item.numpy() returns bytes, and decode() converts bytes to normal strings.

3. Final Answer:

   img1.jpg\nimg2.jpg -> Option C

4. Quick Check:

   Decoded bytes = file names [OK]

Hint: Use .numpy().decode() to get string from tensor [OK]

Common Mistakes:

• Printing tensor directly without decoding
• Expecting list output instead of individual prints
• Confusing bytes and strings

4. Identify the error in the following code snippet that tries to read image files from paths:

import tensorflow as tf
image_paths = ["img1.jpg", "img2.jpg"]
dataset = tf.data.Dataset.from_tensor_slices(image_paths)
dataset = dataset.map(tf.io.read_file)
for img in dataset:
    print(img.numpy().shape)

medium

A. Cannot print shape of a scalar string tensor

B. tf.io.read_file is not a valid function

C. from_tensor_slices requires a tensor, not list

D. map() cannot be used on datasets

Solution

1. Step 1: Analyze dataset after map

   After mapping tf.io.read_file, each element is a scalar string tensor containing raw file bytes.

2. Step 2: Understand tensor shape

   img.numpy() returns Python bytes (raw file content), which has no .shape attribute. Printing img.numpy().shape raises AttributeError.

3. Final Answer:

   Cannot print shape of a scalar string tensor -> Option A

4. Quick Check:

   img.numpy() is bytes; no .shape [OK]

Hint: Raw file bytes are scalars; no shape attribute [OK]

Common Mistakes:

• Assuming read_file returns image tensor
• Thinking from_tensor_slices rejects lists
• Believing map() is invalid on datasets

5. You want to create a TensorFlow dataset from a folder of images, resize each image to 128x128, and batch them in groups of 16. Which code snippet correctly achieves this?

hard

A. dataset = tf.keras.utils.image_dataset_from_directory('images', image_size=(128,128), batch_size=16)

B. dataset = tf.data.Dataset.list_files('images/*').map(lambda x: tf.image.resize(tf.io.decode_image(tf.io.read_file(x)), (128,128))).batch(16)

C. dataset = tf.data.Dataset.from_tensor_slices('images').map(tf.io.read_file).batch(16)

D. dataset = tf.keras.preprocessing.image_dataset_from_directory('images', batch_size=128, image_size=(16,16))

Solution

1. Step 1: Understand dataset creation from folder

   dataset = tf.data.Dataset.list_files('images/*').map(lambda x: tf.image.resize(tf.io.decode_image(tf.io.read_file(x)), (128,128))).batch(16) uses list_files to get file paths, then maps reading, decoding, and resizing images correctly.

2. Step 2: Check batch and resize parameters

   Images are resized to (128,128) and batched in groups of 16 as required.

3. Final Answer:

   dataset = tf.data.Dataset.list_files('images/*').map(lambda x: tf.image.resize(tf.io.decode_image(tf.io.read_file(x)), (128,128))).batch(16) -> Option B

4. Quick Check:

   list_files + map + resize + batch = correct pipeline [OK]

Hint: Use list_files + map with decode and resize, then batch [OK]

Common Mistakes:

• Using wrong batch size or image size parameters
• Confusing keras and tf.data APIs
• Not decoding images before resizing