Kerberos authentication in Hadoop - Step-by-Step Execution

Concept Flow - Kerberos authentication

User requests access

↓

User sends request to KDC

↓

KDC verifies user credentials

↓

KDC issues Ticket Granting Ticket (TGT)

↓

User sends TGT to Ticket Granting Service (TGS)

↓

TGS issues Service Ticket

↓

User presents Service Ticket to Hadoop service

↓

Service verifies ticket and grants access

↓

Access granted

Kerberos authentication uses tickets issued by a trusted server to prove identity and grant access securely.

Execution Sample

Hadoop

1. User requests access
2. KDC issues TGT
3. User requests service ticket
4. TGS issues service ticket
5. User accesses Hadoop service

This sequence shows how a user obtains tickets and accesses a Hadoop service using Kerberos.

Execution Table

Step	Action	Input	Output	Notes
1	User requests access	User credentials	Request sent to KDC	User starts authentication
2	KDC verifies user	Request with credentials	Ticket Granting Ticket (TGT)	TGT proves user identity
3	User requests service ticket	TGT + service name	Request sent to TGS	User asks for service access
4	TGS issues service ticket	Request with TGT	Service Ticket	Ticket for specific Hadoop service
5	User presents service ticket	Service Ticket	Access granted	Hadoop service verifies ticket
6	End	N/A	Authentication complete	User can now use Hadoop securely

💡 Authentication completes after service ticket is verified and access is granted

Variable Tracker

Variable	Start	After Step 2	After Step 4	Final
User Credentials	Provided	Verified by KDC	Used to request service ticket	Not stored after authentication
TGT	None	Issued by KDC	Presented to TGS	Used to get Service Ticket
Service Ticket	None	None	Issued by TGS	Presented to Hadoop service

Key Moments - 3 Insights

Why does the user need a Ticket Granting Ticket (TGT)?

The TGT proves the user's identity to the Ticket Granting Service (TGS) without sending the password again, as shown in steps 2 and 3 of the execution_table.

What happens if the service ticket is invalid or expired?

The Hadoop service will reject access because it cannot verify the ticket, stopping the process at step 5 in the execution_table.

Why is the password not sent to the Hadoop service directly?

Kerberos uses tickets to avoid sending passwords over the network, improving security, as seen in the flow where tickets are exchanged instead of passwords.

Visual Quiz - 3 Questions

Look at the execution_table, what is the output after step 2?

A Ticket Granting Ticket (TGT)

B Service Ticket

C Access granted

D User credentials

💡 Hint

Check the 'Output' column for step 2 in the execution_table.

At which step does the user present the service ticket to the Hadoop service?

A Step 3

B Step 5

C Step 4

D Step 6

💡 Hint

Look for the action involving Hadoop service in the execution_table.

If the KDC does not verify user credentials, what will happen in the flow?

A User gets service ticket directly

B User receives TGT and continues

C User cannot get TGT and authentication stops

D Hadoop service grants access anyway

💡 Hint

Refer to step 2 where KDC verification is critical.

Concept Snapshot

Kerberos authentication uses a trusted server (KDC) to issue tickets.
User first gets a Ticket Granting Ticket (TGT) after verifying credentials.
TGT is used to request service tickets from the Ticket Granting Service (TGS).
Service tickets allow secure access to Hadoop services without sending passwords.
Tickets prove identity and are time-limited for security.

Full Transcript

Kerberos authentication in Hadoop starts when a user requests access by sending credentials to the Key Distribution Center (KDC). The KDC verifies these credentials and issues a Ticket Granting Ticket (TGT). The user then uses this TGT to request a service ticket from the Ticket Granting Service (TGS). Once the service ticket is issued, the user presents it to the Hadoop service. The service verifies the ticket and grants access if valid. This process avoids sending passwords directly and uses time-limited tickets to secure authentication.

Practice

1. What is the main purpose of Kerberos authentication in Hadoop?

easy

A. To manage Hadoop cluster hardware

B. To verify user identities and secure access

C. To store large datasets

D. To speed up data processing

Solution

1. Step 1: Understand Kerberos role in security

   Kerberos is a network authentication protocol that verifies user identities.

2. Step 2: Connect to Hadoop security needs

   Hadoop uses Kerberos to ensure only authorized users access its services.

3. Final Answer:

   To verify user identities and secure access -> Option B

4. Quick Check:

   Kerberos = User identity verification [OK]

Hint: Kerberos = user identity check in Hadoop [OK]

Common Mistakes:

• Confusing Kerberos with data storage
• Thinking it speeds up processing
• Assuming it manages hardware

2. Which command is used to obtain a Kerberos ticket before accessing Hadoop services?

easy

A. kinit

B. klist

C. ktutil

D. hdfs dfs

Solution

1. Step 1: Identify commands related to Kerberos

   Among the options, kinit is used to get a ticket granting ticket (TGT).

2. Step 2: Match command to Hadoop access

   Before accessing Hadoop, users run kinit to authenticate with Kerberos.

3. Final Answer:

   kinit -> Option A

4. Quick Check:

   Get ticket = kinit [OK]

Hint: Use kinit to get Kerberos ticket before Hadoop access [OK]

Common Mistakes:

• Using klist which lists tickets but doesn't get one
• Confusing ktutil which manages keytabs
• Using hdfs dfs which is for file operations

3. What will be the output of the following command if the Kerberos ticket is valid?

klist

medium

A. Lists the current Kerberos tickets with expiration times

B. Deletes all Kerberos tickets

C. Shows Hadoop file system contents

D. Generates a new Kerberos ticket

Solution

1. Step 1: Understand the klist command

   klist displays the current Kerberos tickets held by the user.

2. Step 2: Interpret output when ticket is valid

   If a ticket is valid, klist shows ticket details including expiration times.

3. Final Answer:

   Lists the current Kerberos tickets with expiration times -> Option A

4. Quick Check:

   klist = show tickets [OK]

Hint: klist shows your current Kerberos tickets [OK]

Common Mistakes:

• Thinking klist deletes tickets
• Confusing it with ticket creation
• Assuming it lists Hadoop files

4. You run kinit -kt user.keytab [e-mail] but get an error saying "Client not found in Kerberos database." What is the likely cause?

medium

A. Keytab file is missing

B. Network connection is slow

C. Hadoop service is down

D. Incorrect principal name in the command

Solution

1. Step 1: Analyze the error message

   "Client not found in Kerberos database" means the principal name is not recognized.

2. Step 2: Check command parameters

   If the principal name ([e-mail]) is wrong or misspelled, authentication fails.

3. Final Answer:

   Incorrect principal name in the command -> Option D

4. Quick Check:

   Principal name error = client not found [OK]

Hint: Check principal name spelling if client not found error [OK]

Common Mistakes:

• Assuming keytab file missing causes this error
• Blaming Hadoop service status
• Thinking network speed causes this error

5. You want to automate Hadoop jobs without typing passwords each time. Which method using Kerberos is best suited?

hard

A. Use kinit with manual password entry every time

B. Store passwords in plain text files

C. Use a keytab file with kinit -kt for password-free login

D. Disable Kerberos authentication

Solution

1. Step 1: Understand automation needs

   Automated jobs cannot manually enter passwords each time.

2. Step 2: Identify secure password-free method

   Using a keytab file with kinit -kt allows password-free authentication securely.

3. Step 3: Evaluate other options

   Manual entry is not automation-friendly; storing passwords in plain text is insecure; disabling Kerberos removes security.

4. Final Answer:

   Use a keytab file with kinit -kt for password-free login -> Option C

5. Quick Check:

   Keytab + kinit -kt = automated login [OK]

Hint: Use keytab with kinit -kt for automation [OK]

Common Mistakes:

• Trying manual password entry for automation
• Storing passwords insecurely
• Disabling security features