Hadoop vs Spark comparison - Quick Revision & Key Differences

Recall & Review

beginner

What is Hadoop primarily used for?

Hadoop is mainly used for storing and processing large data sets using a distributed file system called HDFS and batch processing with MapReduce.

beginner

How does Spark process data differently from Hadoop MapReduce?

Spark processes data in-memory, which makes it faster for iterative tasks and real-time processing compared to Hadoop MapReduce that reads and writes data to disk between steps.

beginner

Which system supports real-time data processing better: Hadoop or Spark?

Spark supports real-time data processing better because it can handle streaming data and perform computations quickly using in-memory processing.

beginner

What is the main storage system used by Hadoop?

Hadoop uses HDFS (Hadoop Distributed File System) to store data across many machines in a cluster.

intermediate

Why might Spark be preferred over Hadoop for machine learning tasks?

Spark is preferred because it can quickly process data in-memory and has built-in libraries like MLlib for machine learning, making it faster and easier to use for iterative algorithms.

Which of the following is a key feature of Spark compared to Hadoop?

A Only batch processing

B Uses MapReduce only

C In-memory data processing

D No support for streaming data

Spark processes data in-memory, which makes it faster than Hadoop's disk-based MapReduce.

What does Hadoop primarily use to store data?

A Spark SQL

B In-memory cache

C NoSQL databases

D HDFS

Hadoop stores data in HDFS, a distributed file system.

Which system is better suited for real-time data processing?

A Hadoop MapReduce

B Spark

C Both are equal

D Neither supports real-time

Spark supports real-time streaming and in-memory processing, making it better for real-time tasks.

Which processing model does Hadoop MapReduce follow?

A Batch processing

B Streaming processing

C In-memory processing

D Graph processing

Hadoop MapReduce processes data in batches, reading and writing to disk between steps.

Why is Spark faster than Hadoop MapReduce for iterative algorithms?

A Because it processes data in-memory

B Because it uses disk storage

C Because it uses MapReduce

D Because it does not support machine learning

Spark keeps data in memory during processing, speeding up iterative tasks like machine learning.

Explain the main differences between Hadoop and Spark in terms of data processing and speed.

Think about how each system handles data during processing.

Describe scenarios where you would choose Hadoop over Spark and vice versa.

Consider the type of data processing and speed requirements.