Output modes (append, complete, update) in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to set the output mode to append in a streaming query.

Apache Spark

query = df.writeStream.outputMode('[1]').start()

Drag options to blanks, or click blank then click option'

Acomplete

Bappend

Cupdate

Doverwrite

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'complete' mode when only new rows should be added.

Using 'update' mode which updates existing rows instead of appending.

The append mode adds only new rows to the output sink.

2fill in blank

medium

Complete the code to set the output mode to complete in a streaming query.

Apache Spark

query = df.writeStream.outputMode('[1]').start()

Drag options to blanks, or click blank then click option'

Acomplete

Bupdate

Coverwrite

Dappend

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'append' mode which only adds new rows.

Using 'update' mode which only outputs changed rows.

The complete mode writes the entire result table to the sink every time there is an update.

3fill in blank

hard

Fix the error in setting the output mode to update in a streaming query.

Apache Spark

query = df.writeStream.outputMode('[1]').start()

Drag options to blanks, or click blank then click option'

Aappend

Breplace

Ccomplete

Dupdate

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'replace' which is not a valid output mode.

Using 'append' or 'complete' when update is needed.

The update mode outputs only the rows that have changed since the last trigger.

4fill in blank

hard

Fill both blanks to create a streaming query that writes to console with update mode and no truncation.

Apache Spark

query = df.writeStream.format('[1]').outputMode('[2]').option('truncate', false).start()

Drag options to blanks, or click blank then click option'

Aconsole

Bappend

Cupdate

Dmemory

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'append' mode with console when update is required.

Using 'memory' format which stores data in memory, not console.

The console format outputs to the console, and update mode outputs only changed rows.

5fill in blank

hard

Fill all three blanks to write a streaming DataFrame to memory with complete mode and query name 'myQuery'.

Apache Spark

query = df.writeStream.format('[1]').outputMode('[2]').queryName('[3]').start()

Drag options to blanks, or click blank then click option'

Amemory

Bcomplete

CmyQuery

Dappend

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'append' mode with memory format which is not supported.

Forgetting to name the query with queryName.

Writing to memory allows querying the result table, complete mode outputs full results, and myQuery names the query.