Alternatives for big data (Datashader, HoloViews) in Matplotlib - Mini Project: Build & Apply

Visualizing Large Data with Datashader and HoloViews

📖 Scenario: You work as a data analyst for a city planning team. They have collected millions of GPS points from taxis to understand traffic patterns. Plotting all points with normal tools is slow and unclear. You will learn how to use Datashader and HoloViews to visualize big data efficiently.

🎯 Goal: Create a large dataset of GPS points, set a plotting configuration, use Datashader with HoloViews to visualize the data, and display the result.

📋 What You'll Learn

Create a dictionary called taxi_data with keys 'x' and 'y' containing lists of 100000 random float values between 0 and 100

Create a variable called plot_width and set it to 600

Use hv.Points with taxi_data and apply datashade with the width set to plot_width

Print the resulting HoloViews object

💡 Why This Matters

🌍 Real World

City planners and data scientists often work with huge GPS datasets to understand traffic and movement patterns. Visualizing millions of points quickly helps make better decisions.

💼 Career

Skills in Datashader and HoloViews are valuable for data scientists and analysts working with big data visualization, especially in urban planning, transportation, and geospatial analysis.

1

Create the large GPS dataset

Create a dictionary called taxi_data with keys 'x' and 'y'. Each key should map to a list of 100000 random float values between 0 and 100. Use numpy.random.uniform(0, 100, 100000).tolist() for both lists.

Matplotlib

import numpy as np
# Create the taxi_data dictionary with 'x' and 'y' keys containing 100000 random floats between 0 and 100
# Your code here

Hint

Use np.random.uniform(0, 100, 100000).tolist() to generate the lists for both 'x' and 'y'.

2

Set the plot width configuration

Create a variable called plot_width and set it to 600.

Matplotlib

import numpy as np
taxi_data = {
    'x': np.random.uniform(0, 100, 100000).tolist(),
    'y': np.random.uniform(0, 100, 100000).tolist()
}
# Set plot_width to 600
# Your code here

Hint

Just assign the number 600 to the variable plot_width.

3

Create the Datashader plot with HoloViews

Import holoviews as hv. Then import datashade from holoviews.operation.datashader. Initialize HoloViews with hv.extension('bokeh'). Create a hv.Points object called points using taxi_data. Use datashade(points, width=plot_width) to create a shaded plot called shaded_plot.

Matplotlib

import numpy as np
taxi_data = {
    'x': np.random.uniform(0, 100, 100000).tolist(),
    'y': np.random.uniform(0, 100, 100000).tolist()
}
plot_width = 600

# Import holoviews as hv
# from holoviews.operation.datashader import datashade
# Initialize hv extension with 'bokeh'
# Create hv.Points called points from taxi_data
# Use datashade on points with width=plot_width and assign to shaded_plot
# Your code here

Hint

Remember to import datashade from holoviews.operation.datashader and initialize HoloViews with hv.extension('bokeh').

4

Display the shaded plot

Print the shaded_plot object to display the visualization.

Matplotlib

import numpy as np
import holoviews as hv
from holoviews.operation.datashader import datashade

hv.extension('bokeh')

taxi_data = {
    'x': np.random.uniform(0, 100, 100000).tolist(),
    'y': np.random.uniform(0, 100, 100000).tolist()
}
plot_width = 600

points = hv.Points(taxi_data)
shaded_plot = datashade(points, width=plot_width)

# Print shaded_plot to display the result
# Your code here

Hint

Use print(shaded_plot) to show the plot object. The exact output may vary but should include the HoloViews object type.

Practice

1. What is the main advantage of using Datashader or HoloViews over standard Matplotlib for big data visualization?

easy

A. They efficiently handle and visualize very large datasets without slowing down.

B. They produce 3D plots automatically.

C. They require less memory for small datasets.

D. They only work with time series data.

Solution

1. Step 1: Understand the challenge with big data in Matplotlib

   Standard Matplotlib struggles with very large datasets because plotting millions of points slows down rendering and makes plots unclear.

2. Step 2: Identify the benefit of Datashader and HoloViews

   Datashader and HoloViews use smart techniques to aggregate and render large data quickly and clearly, making visualization efficient.

3. Final Answer:

   They efficiently handle and visualize very large datasets without slowing down. -> Option A

4. Quick Check:

   Big data visualization = Efficient handling [OK]

Hint: Big data needs tools that handle millions of points fast [OK]

Common Mistakes:

• Thinking they only create 3D plots
• Assuming they reduce memory for small data
• Believing they work only with time series

2. Which of the following is the correct way to import Datashader and HoloViews in Python?

easy

A. import datashader as ds; import holoviews as hv

B. import datashader; import holoviews.plot

C. from matplotlib import datashader, holoviews

D. import ds; import hv

Solution

1. Step 1: Recall standard import syntax for these libraries

   Datashader is usually imported as 'import datashader as ds' and HoloViews as 'import holoviews as hv' for convenience.

2. Step 2: Check each option for correctness

   import datashader as ds; import holoviews as hv uses correct import statements. import datashader; import holoviews.plot tries to import a submodule incorrectly. from matplotlib import datashader, holoviews wrongly imports from matplotlib. import ds; import hv uses undefined aliases without import.

3. Final Answer:

   import datashader as ds; import holoviews as hv -> Option A

4. Quick Check:

   Standard imports = import datashader as ds; import holoviews as hv [OK]

Hint: Use 'import library as alias' for common big data libs [OK]

Common Mistakes:

• Trying to import from matplotlib
• Using undefined aliases without import
• Importing submodules incorrectly

3. Given the code below, what will be the output type when using Datashader with HoloViews?

import datashader as ds
import holoviews as hv
import pandas as pd

hv.extension('bokeh')
data = pd.DataFrame({'x': range(1000000), 'y': range(1000000)})
points = ds.Points(data, 'x', 'y')
shaded = ds.Canvas().shade(points)
print(type(shaded))

medium

A. <class 'pandas.core.frame.DataFrame'>

B. <class 'holoviews.core.element.Points'>

C. <class 'matplotlib.figure.Figure'>

D. <class 'datashader.transfer_functions.Image'>

Solution

1. Step 1: Understand what ds.Canvas().shade() returns

   The shade() function in Datashader returns an Image object representing the rasterized plot.

2. Step 2: Check the printed type

   Since shade() returns a datashader.transfer_functions.Image object, the printed type matches <class 'datashader.transfer_functions.Image'>.

3. Final Answer:

   <class 'datashader.transfer_functions.Image'> -> Option D

4. Quick Check:

   Datashader shade output = Image object [OK]

Hint: shade() returns an Image object, not raw data [OK]

Common Mistakes:

• Thinking shade returns raw DataFrame
• Confusing HoloViews Points with shaded image
• Expecting a Matplotlib figure object

4. Identify the error in the following code snippet using HoloViews and Datashader:

import holoviews as hv
import datashader as ds
hv.extension('bokeh')
data = {'x': [1,2,3], 'y': [4,5,6]}
points = hv.Points(data)
canvas = ds.Canvas()
img = canvas.shade(points)
img

medium

A. shade() method does not exist in Canvas class.

B. Missing import for pandas library.

C. ds.Canvas().shade() expects a Datashader Element (e.g. ds.Points), not a HoloViews Points object.

D. hv.extension('bokeh') should be called after creating points.

Solution

1. Step 1: Check source passed to ds.Canvas().shade()

   ds.Canvas().shade() requires a Datashader Element like ds.Points(), but points is an hv.Points object, which is incompatible.

2. Step 2: Confirm other code parts

   Dict data is fine for hv.Points(); no pandas needed; shade() exists; extension() can be called anytime.

3. Final Answer:

   ds.Canvas().shade() expects a Datashader Element (e.g. ds.Points), not a HoloViews Points object. -> Option C

4. Quick Check:

   ds.Canvas.shade needs ds.Element [OK]

Hint: ds.Canvas.shade requires Datashader Element, not HoloViews Points [OK]

Common Mistakes:

• Thinking dict data is invalid for hv.Points
• Believing shade() method is missing
• Assuming extension order causes the error

5. You have a dataset with 10 million points and want to create an interactive plot that updates quickly when zooming. Which approach best uses Datashader and HoloViews together?

hard

A. Plot all points directly with Matplotlib scatter for best performance.

B. Use HoloViews Points with Datashader's dynamic rasterization and link it to a Bokeh plot for interactivity.

C. Convert data to a small sample and plot with HoloViews only.

D. Use Datashader to create static PNG images and display them without interactivity.

Solution

1. Step 1: Understand the need for interactivity with big data

   Plotting 10 million points directly is slow; dynamic rasterization lets you update plots quickly on zoom.

2. Step 2: Identify the best integration method

   HoloViews with Datashader supports dynamic rasterization and can link to Bokeh for interactive zoom and pan, making it ideal.

3. Final Answer:

   Use HoloViews Points with Datashader's dynamic rasterization and link it to a Bokeh plot for interactivity. -> Option B

4. Quick Check:

   Dynamic rasterization + Bokeh = Fast interactive big data plots [OK]

Hint: Combine Datashader + HoloViews + Bokeh for big interactive plots [OK]

Common Mistakes:

• Trying to plot all points directly in Matplotlib
• Using only small samples losing data detail
• Creating static images without interactivity