Installing Airflow locally - Visual Walkthrough

Process Flow - Installing Airflow locally

Start: Prepare environment

↓

Create virtual environment

↓

Activate virtual environment

↓

Install Airflow with pip

↓

Initialize Airflow database

↓

Start Airflow webserver

↓

Start Airflow scheduler

↓

Access Airflow UI in browser

↓

Done

This flow shows the step-by-step process to install and run Airflow locally using a virtual environment.

Execution Sample

Apache Airflow

python3 -m venv airflow_env
source airflow_env/bin/activate
pip install apache-airflow
airflow db init
airflow webserver -p 8080
airflow scheduler

Commands to create a virtual environment, install Airflow, initialize its database, and start the webserver and scheduler.

Process Table

Step	Command	Action	Result/Output
1	python3 -m venv airflow_env	Create virtual environment folder	Folder 'airflow_env' created with isolated Python environment
2	source airflow_env/bin/activate	Activate virtual environment	Shell prompt changes, Python points to 'airflow_env'
3	pip install apache-airflow	Install Airflow package	Airflow and dependencies installed in virtual environment
4	airflow db init	Initialize Airflow metadata database	Database tables created, ready for Airflow use
5	airflow webserver -p 8080	Start Airflow webserver on port 8080	Webserver running, logs show startup success
6	airflow scheduler	Start Airflow scheduler	Scheduler running, ready to execute workflows
7	Access http://localhost:8080	Open Airflow UI in browser	Airflow dashboard loads, showing status and DAGs
8	Exit	Stop commands or close terminal	Airflow services stop, environment deactivated

💡 All steps completed successfully, Airflow is installed and running locally.

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	Final
Virtual Environment	None	Created 'airflow_env'	Activated 'airflow_env'	Active 'airflow_env'	Active 'airflow_env'	Active 'airflow_env'	Active 'airflow_env'	Active 'airflow_env'
Airflow Package	Not installed	Not installed	Installed	Installed	Installed	Installed	Installed	Installed
Airflow DB	Not initialized	Not initialized	Not initialized	Initialized	Initialized	Initialized	Initialized	Initialized
Webserver	Not running	Not running	Not running	Not running	Running	Running	Running	Running
Scheduler	Not running	Not running	Not running	Not running	Not running	Not running	Running	Running

Key Moments - 3 Insights

Why do we create and activate a virtual environment before installing Airflow?

Creating and activating a virtual environment isolates Airflow and its dependencies from other Python projects, preventing conflicts. This is shown in steps 1 and 2 of the execution_table.

What does 'airflow db init' do and why is it necessary?

'airflow db init' sets up the database tables Airflow needs to track workflows and tasks. Without this, Airflow cannot run properly. This is step 4 in the execution_table.

Why do we start both the webserver and the scheduler?

The webserver provides the user interface to monitor workflows, while the scheduler runs the workflows on time. Both are needed for Airflow to function fully, as seen in steps 5 and 6.

Visual Quiz - 3 Questions

Look at the execution_table, after which step is the Airflow package installed?

A After step 2

B After step 3

C After step 4

D After step 5

💡 Hint

Check the 'Airflow Package' row in variable_tracker after step 3.

At which step does the Airflow webserver start running?

A Step 5

B Step 6

C Step 4

D Step 7

💡 Hint

Look at the 'Webserver' variable in variable_tracker and the execution_table step 5.

If you skip activating the virtual environment, what would likely happen when installing Airflow?

A The installation fails with an error

B Airflow installs in the virtual environment anyway

C Airflow installs globally, possibly causing conflicts

D Nothing happens, installation is skipped

💡 Hint

Refer to key_moments about virtual environment purpose and step 2 activation.

Concept Snapshot

Install Airflow locally:
1. Create virtual env: python3 -m venv airflow_env
2. Activate it: source airflow_env/bin/activate
3. Install Airflow: pip install apache-airflow
4. Initialize DB: airflow db init
5. Start webserver: airflow webserver -p 8080
6. Start scheduler: airflow scheduler
Use http://localhost:8080 to access UI.

Full Transcript

To install Airflow locally, first create a virtual environment to keep dependencies isolated. Activate it to use that environment. Then install Airflow using pip inside it. Initialize the Airflow database to prepare metadata storage. Start the Airflow webserver to access the UI and the scheduler to run workflows. Finally, open the browser at localhost port 8080 to see the Airflow dashboard. This process ensures Airflow runs cleanly on your machine without affecting other Python projects.

Practice

1. What is the first step to install Apache Airflow locally on your machine?

easy

A. Create an Airflow user

B. Start the Airflow webserver

C. Create and activate a Python virtual environment

D. Initialize the Airflow database

Solution

1. Step 1: Set up isolated Python environment

   Creating and activating a virtual environment ensures Airflow and its dependencies do not interfere with other Python projects.

2. Step 2: Prepare for Airflow installation

   Once the virtual environment is active, you can safely install Airflow using pip.

3. Final Answer:

   Create and activate a Python virtual environment -> Option C

4. Quick Check:

   First step = virtual environment setup [OK]

Hint: Always start with a virtual environment for Python projects [OK]

Common Mistakes:

• Skipping virtual environment and installing globally
• Trying to start Airflow before installation
• Initializing database before installing Airflow

2. Which command correctly installs Apache Airflow with pip inside a virtual environment?

easy

A. pip install airflow

B. pip install airflow-webserver

C. pip install airflow==latest

D. pip install apache-airflow

Solution

1. Step 1: Use the official package name

   The correct package name for Airflow on PyPI is 'apache-airflow'.

2. Step 2: Avoid incorrect or incomplete package names

   Other options either use wrong names or non-existent packages.

3. Final Answer:

   pip install apache-airflow -> Option D

4. Quick Check:

   Correct package name = apache-airflow [OK]

Hint: Use 'apache-airflow' as the pip package name [OK]

Common Mistakes:

• Using 'airflow' instead of 'apache-airflow'
• Adding invalid version specifiers
• Trying to install non-existent sub-packages

3. After installing Airflow, what is the output of running airflow db init?

medium

A. Initializes the Airflow metadata database

B. Creates a new Airflow user

C. Starts the Airflow webserver

D. Displays Airflow version information

Solution

1. Step 1: Understand the command purpose

   The command 'airflow db init' sets up the metadata database needed for Airflow to track workflows and tasks.

2. Step 2: Differentiate from other commands

   Starting the webserver, creating users, or showing version info are done with different commands.

3. Final Answer:

   Initializes the Airflow metadata database -> Option A

4. Quick Check:

   'airflow db init' = database initialization [OK]

Hint: Init command sets up Airflow's database, not the server [OK]

Common Mistakes:

• Confusing db init with webserver start
• Expecting user creation from db init
• Thinking it shows version info

4. You run airflow webserver but get an error saying the database is not initialized. What should you do to fix this?

medium

A. Run airflow db init to initialize the database

B. Create a new Airflow user first

C. Restart the machine and try again

D. Install Airflow again using pip

Solution

1. Step 1: Identify the cause of the error

   The error indicates the Airflow metadata database is missing or not set up.

2. Step 2: Initialize the database

   Running 'airflow db init' creates the necessary database tables and schema.

3. Final Answer:

   Run airflow db init to initialize the database -> Option A

4. Quick Check:

   Database error fixed by db init command [OK]

Hint: Initialize database before starting webserver [OK]

Common Mistakes:

• Skipping database initialization
• Trying to create user before db init
• Restarting machine without fixing db

5. Which sequence of commands correctly installs Airflow, initializes the database, creates an admin user, and starts the webserver and scheduler locally?

hard

A. pip install apache-airflow; airflow users create --username admin; airflow db init; airflow webserver; airflow scheduler

B. python -m venv venv; source venv/bin/activate; pip install apache-airflow; airflow db init; airflow users create --username admin --firstname Admin --lastname User --role Admin --email [e-mail] --password admin; airflow webserver & airflow scheduler

C. source venv/bin/activate; airflow db init; pip install apache-airflow; airflow webserver; airflow scheduler

D. python -m venv venv; pip install apache-airflow; airflow webserver; airflow db init; airflow scheduler

Solution

1. Step 1: Set up virtual environment and install Airflow

   Create and activate a virtual environment, then install Airflow with pip.

2. Step 2: Initialize database and create admin user

   Run 'airflow db init' to prepare the database, then create an admin user with the correct command.

3. Step 3: Start Airflow services

   Start the webserver and scheduler to run Airflow locally.

4. Final Answer:

   python -m venv venv; source venv/bin/activate; pip install apache-airflow; airflow db init; airflow users create --username admin --firstname Admin --lastname User --role Admin --email [e-mail] --password admin; airflow webserver & airflow scheduler -> Option B

5. Quick Check:

   Correct full setup sequence = python -m venv venv; source venv/bin/activate; pip install apache-airflow; airflow db init; airflow users create --username admin --firstname Admin --lastname User --role Admin --email [e-mail] --password admin; airflow webserver & airflow scheduler [OK]

Hint: Follow install, init, user create, then start services [OK]

Common Mistakes:

• Installing Airflow after db init
• Starting webserver before creating user
• Not activating virtual environment