NiFi for data flow automation in Hadoop - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding NiFi FlowFiles

In Apache NiFi, what is a FlowFile?

A A container that holds data and attributes moving through NiFi

B A data record stored in HDFS for batch processing

C A script that automates data ingestion in NiFi

D A configuration file for NiFi processors

💡 Hint

Think about what moves inside NiFi between processors.

A FlowFile is the basic data packet in NiFi. It contains the actual data content and metadata attributes, allowing NiFi to track and route data through its flow.

❓ Predict Output

intermediate

NiFi Expression Language Output

What is the output of this NiFi Expression Language snippet if the attribute 'filename' is 'data_2024.csv'?

${filename:substring(5,9)}

A 024.

B 2024

C 202

D data

💡 Hint

Substring(start, end) extracts characters from start index up to but not including end index.

The substring from index 5 to 9 in 'data_2024.csv' is '2024'. Indexing starts at 0.

❓ data_output

advanced

NiFi Data Provenance Query Result

You run a NiFi Data Provenance query filtering events by a specific processor and time range. The query returns 3 events. What does this output represent?

A Three processors active during the time range

B Three configuration changes made to NiFi

C Three errors logged by NiFi in that time range

D Three FlowFiles processed by that processor in the time range

💡 Hint

Data Provenance tracks FlowFile events like creation, modification, and transfer.

The query output shows FlowFile events processed by the selected processor during the specified time.

🔧 Debug

advanced

Troubleshooting NiFi Processor Failure

Given this NiFi processor configuration snippet, which option explains why the processor fails to start?

Properties:
Input Directory: /data/input
File Filter: *.csv
Scheduling Strategy: Timer-driven
Run Schedule: 0 sec

A Input Directory path must be absolute, '/data/input' is relative

B File Filter '*.csv' is invalid syntax and causes failure

C Run Schedule set to 0 seconds causes invalid scheduling interval error

D Scheduling Strategy 'Timer-driven' is deprecated and causes failure

💡 Hint

Check if the run schedule value is valid for timer-driven scheduling.

Run Schedule cannot be zero seconds; it must be a positive number to schedule processor runs.

🚀 Application

expert

Designing a NiFi Flow for Real-Time Data Filtering

You need to design a NiFi flow that reads streaming data from Kafka, filters records where 'status' equals 'error', and writes them to HDFS. Which sequence of processors correctly implements this?

A ConsumeKafka -> QueryRecord (filter status=='error') -> PutHDFS

B GetFile -> UpdateAttribute (status=='error') -> PutHDFS

C ConsumeKafka -> RouteOnAttribute (status=='error') -> PutHDFS

D ListenHTTP -> RouteOnContent (status=='error') -> PutFile

💡 Hint

Consider processors that can consume Kafka and filter records by content.

ConsumeKafka reads streaming data, QueryRecord filters records by field values, and PutHDFS writes filtered data to HDFS.