HDFS command line interface in Hadoop - Mini Project: Build & Apply

HDFS Command Line Interface Basics

📖 Scenario: You are working with big data stored in Hadoop Distributed File System (HDFS). You need to manage files and directories using the HDFS command line interface (CLI).

🎯 Goal: Learn to use basic HDFS CLI commands to create directories, upload files, list contents, and check file details.

📋 What You'll Learn

Use hdfs dfs -mkdir to create directories

Use hdfs dfs -put to upload files

Use hdfs dfs -ls to list directory contents

Use hdfs dfs -cat to display file contents

💡 Why This Matters

🌍 Real World

Managing big data files in HDFS is essential for data engineers and analysts working with Hadoop ecosystems.

💼 Career

Knowing HDFS CLI commands helps in daily tasks like data ingestion, organization, and inspection in big data jobs.

1

Create a directory in HDFS

Write the command to create a directory called /user/data in HDFS using hdfs dfs -mkdir.

Hadoop

# Your command here

Need a hint?

Use hdfs dfs -mkdir followed by the directory path.

2

Upload a file to HDFS directory

Write the command to upload a local file called sales.csv to the HDFS directory /user/data using hdfs dfs -put.

Hadoop

hdfs dfs -mkdir /user/data
# Your command here

Need a hint?

Use hdfs dfs -put followed by the local file and the HDFS directory.

3

List files in HDFS directory

Write the command to list all files in the HDFS directory /user/data using hdfs dfs -ls.

Hadoop

hdfs dfs -mkdir /user/data
hdfs dfs -put sales.csv /user/data
# Your command here

Need a hint?

Use hdfs dfs -ls followed by the directory path.

4

Display contents of a file in HDFS

Write the command to display the contents of the file /user/data/sales.csv using hdfs dfs -cat.

Hadoop

hdfs dfs -mkdir /user/data
hdfs dfs -put sales.csv /user/data
hdfs dfs -ls /user/data
# Your command here

Need a hint?

Use hdfs dfs -cat followed by the file path.