Hadoop - Modern Data Architecture with Hadoop
In a Kappa architecture using Kafka and Spark, what will happen if the streaming job is restarted without checkpointing?
In a Kappa architecture using Kafka and Spark, what will happen if the streaming job is restarted without checkpointing?
medium📝 Predict Output Q5 of 15
Step-by-Step Solution
Solution:
Step 1: Understand checkpoint role in streaming
Checkpoint stores offsets to resume processing from last point.Step 2: Effect of missing checkpoint on restart
Without checkpoint, Spark starts from default "latest" offset, processing only new messages.Final Answer:
The job will skip all data and process only new messages -> Option BQuick Check:
No checkpoint means start from latest offset [OK]
Quick Trick: No checkpoint = process only new messages [OK]
Common Mistakes:
- Thinking it resumes from last offset
- Believing job fails without checkpoint
- Assuming it starts from earliest offset
Master "Modern Data Architecture with Hadoop" in Hadoop
Want More Practice?
15+ quiz questions · All difficulty levels · Free
Free Signup - Practice All QuestionsMore Hadoop Quizzes