Creating databases and tables in Hadoop - Performance & Efficiency

Time Complexity: Creating databases and tables

O(n)

Understanding Time Complexity

When we create databases and tables in Hadoop, we want to know how long it takes as we add more commands or data.

We ask: How does the time to create these structures grow when we create more of them?

Scenario Under Consideration

Analyze the time complexity of the following code snippet.

CREATE DATABASE IF NOT EXISTS sales_db;
USE sales_db;
CREATE TABLE IF NOT EXISTS transactions (
  id INT,
  amount FLOAT,
  date STRING
) STORED AS PARQUET;

CREATE TABLE IF NOT EXISTS customers (
  customer_id INT,
  name STRING
) STORED AS ORC;

This code creates one database and two tables inside it, defining their structure and storage format.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: Executing each CREATE statement one by one.
• How many times: Once per database or table creation command.

How Execution Grows With Input

Each new database or table requires a separate command to run.

Input Size (n)	Approx. Operations
10	10 commands executed
100	100 commands executed
1000	1000 commands executed

Pattern observation: The time grows directly with the number of create commands.

Final Time Complexity

Time Complexity: O(n)

This means the time to create databases and tables grows in a straight line as you add more create commands.

Common Mistake

[X] Wrong: "Creating multiple tables happens all at once, so time stays the same no matter how many tables."

[OK] Correct: Each create command runs separately, so more tables mean more commands and more time.

Interview Connect

Understanding how time grows with commands helps you explain system behavior clearly and shows you think about efficiency in real projects.

Self-Check

"What if we batch multiple table creations into a single script? How would the time complexity change?"

Practice

1. What is the main purpose of creating a database in Hadoop?

easy

A. To run queries faster

B. To store data in rows and columns

C. To organize data into separate groups

D. To delete old data automatically

Solution

1. Step 1: Understand the role of a database

   A database is used to organize data logically into separate groups or collections.

2. Step 2: Differentiate database from tables

   Tables store data in rows and columns inside a database, but the database itself is for grouping.

3. Final Answer:

   To organize data into separate groups -> Option C

4. Quick Check:

   Database = Organize data groups [OK]

Hint: Databases group data; tables hold the data [OK]

Common Mistakes:

• Confusing database with table
• Thinking database stores data directly
• Assuming database speeds up queries automatically

2. Which of the following is the correct syntax to create a database named sales_db in Hive?

easy

A. CREATE DATABASE sales_db;

B. MAKE DATABASE sales_db;

C. CREATE DB sales_db;

D. NEW DATABASE sales_db;

Solution

1. Step 1: Recall Hive syntax for database creation

   The correct command to create a database in Hive is CREATE DATABASE database_name;.

2. Step 2: Check each option

   Only CREATE DATABASE sales_db; matches the correct syntax exactly.

3. Final Answer:

   CREATE DATABASE sales_db; -> Option A

4. Quick Check:

   CREATE DATABASE = correct syntax [OK]

Hint: Use CREATE DATABASE, not MAKE or NEW [OK]

Common Mistakes:

• Using MAKE or NEW instead of CREATE
• Abbreviating DATABASE as DB
• Missing semicolon at end

3. What will be the result of running this Hive query?

CREATE TABLE employees (id INT, name STRING, salary FLOAT);

medium

A. Creates a table named employees with three columns

B. Throws a syntax error due to missing database name

C. Creates a database named employees

D. Creates a table with no columns

Solution

1. Step 1: Analyze the CREATE TABLE statement

   The query uses correct syntax to create a table named employees with columns id, name, and salary.

2. Step 2: Check for errors or missing parts

   Database name is optional if default database is used; no syntax error here.

3. Final Answer:

   Creates a table named employees with three columns -> Option A

4. Quick Check:

   CREATE TABLE with columns = creates table [OK]

Hint: CREATE TABLE defines columns inside parentheses [OK]

Common Mistakes:

• Thinking database name is mandatory
• Confusing table and database creation
• Assuming syntax error without checking carefully

4. Identify the error in this Hive query:

CREATE TABLE users id INT, name STRING;

medium

A. Table name is invalid

B. Missing parentheses around column definitions

C. Data types are incorrect

D. CREATE TABLE keyword is misspelled

Solution

1. Step 1: Review the syntax for CREATE TABLE

   Column definitions must be enclosed in parentheses after the table name.

2. Step 2: Check the given query

   The query misses parentheses around id INT, name STRING, causing syntax error.

3. Final Answer:

   Missing parentheses around column definitions -> Option B

4. Quick Check:

   Columns need parentheses [OK]

Hint: Always use parentheses for columns in CREATE TABLE [OK]

Common Mistakes:

• Forgetting parentheses
• Assuming table name is wrong
• Thinking data types are invalid

5. You want to create a database company_db and inside it a table departments with columns dept_id INT and dept_name STRING. Which sequence of commands is correct?

hard

A. CREATE TABLE company_db.departments (dept_id INT, dept_name STRING);

B. CREATE DATABASE company_db; CREATE TABLE departments dept_id INT, dept_name STRING;

C. USE company_db; CREATE DATABASE company_db; CREATE TABLE departments (dept_id INT, dept_name STRING);

D. CREATE DATABASE company_db; USE company_db; CREATE TABLE departments (dept_id INT, dept_name STRING);

Solution

1. Step 1: Create the database first

   Use CREATE DATABASE company_db; to make the database.

2. Step 2: Switch to the new database

   Use USE company_db; to set the context for table creation.

3. Step 3: Create the table with correct syntax

   Use CREATE TABLE departments (dept_id INT, dept_name STRING); inside the database.

4. Final Answer:

   CREATE DATABASE company_db; USE company_db; CREATE TABLE departments (dept_id INT, dept_name STRING); -> Option D

5. Quick Check:

   Create DB, switch DB, then create table [OK]

Hint: Create DB, then USE it before creating tables [OK]

Common Mistakes:

• Trying to create table before database
• Wrong order of USE and CREATE DATABASE
• Missing parentheses in CREATE TABLE