Kerberos authentication in Hadoop - Mini Project: Build & Apply

Kerberos Authentication Setup for Hadoop

📖 Scenario: You are working in a company that uses Hadoop to store and process big data. To keep the data safe, the company uses Kerberos authentication. You need to set up a simple Kerberos authentication configuration for Hadoop to understand how it works.

🎯 Goal: Build a basic Kerberos authentication setup by creating a keytab file, configuring Hadoop to use Kerberos, and verifying the authentication works.

📋 What You'll Learn

Create a Kerberos principal for Hadoop service

Generate a keytab file for the principal

Configure Hadoop core-site.xml to use Kerberos authentication

Verify Kerberos authentication by obtaining a ticket and listing Hadoop files

💡 Why This Matters

🌍 Real World

Kerberos authentication is widely used in big data environments to secure access to Hadoop clusters and protect sensitive data.

💼 Career

Understanding Kerberos setup is important for data engineers and administrators managing secure Hadoop deployments.

1

Create a Kerberos principal for Hadoop service

Use the kadmin.local command to create a Kerberos principal called hadoop/[e-mail] with a password.

Hadoop

# Use kadmin.local to create the principal
# Your code here

Need a hint?

Use kadmin.local -q "addprinc -randkey hadoop/[e-mail]" to create the principal with a random key.

2

Generate a keytab file for the Hadoop principal

Use the kadmin.local command to export the keytab file named hadoop.service.keytab for the principal hadoop/[e-mail].

Hadoop

kadmin.local -q "addprinc -randkey hadoop/[e-mail]"
# Export the keytab file below
# Your code here

Need a hint?

Use ktadd -k hadoop.service.keytab hadoop/[e-mail] to create the keytab file.

3

Configure Hadoop to use Kerberos authentication

Edit the core-site.xml file to add the property hadoop.security.authentication with the value kerberos.

Hadoop

kadmin.local -q "addprinc -randkey hadoop/[e-mail]"
kadmin.local -q "ktadd -k hadoop.service.keytab hadoop/[e-mail]"
# Add the following XML snippet to core-site.xml
# Your code here

Need a hint?

Add the property inside core-site.xml to enable Kerberos authentication.

4

Verify Kerberos authentication by obtaining a ticket and listing Hadoop files

Use kinit -kt hadoop.service.keytab hadoop/[e-mail] to get a Kerberos ticket, then run hdfs dfs -ls / to list files in HDFS.

Hadoop

kadmin.local -q "addprinc -randkey hadoop/[e-mail]"
kadmin.local -q "ktadd -k hadoop.service.keytab hadoop/[e-mail]"

# core-site.xml snippet:
# 
#   
#     hadoop.security.authentication
#     kerberos
#   
# 

# Run kinit and list HDFS files below
# Your code here

Need a hint?

Use kinit with the keytab to get a ticket, then list files with hdfs dfs -ls /.