Environment management with conda and pip in MLOps - Commands & Configuration

Introduction

Managing software environments helps keep your projects organized and avoids conflicts between different package versions. Conda and pip are tools that let you create and control these environments easily.

When you want to keep different projects isolated so their packages don't interfere with each other

When you need to install specific versions of Python or libraries for a project

When you want to share your environment setup with teammates or on another computer

When you want to add or update packages without breaking your existing setup

When you want to switch quickly between different project environments

Config File - environment.yml

environment.yml

name: my-ml-project
dependencies:
  - python=3.9
  - numpy=1.23.1
  - pandas=1.5.2
  - pip:
    - scikit-learn==1.1.3
    - matplotlib==3.6.2

This environment.yml file defines a conda environment named my-ml-project. It specifies Python 3.9 and some packages managed by conda like numpy and pandas. Under pip, it lists packages installed via pip, such as scikit-learn and matplotlib. This file lets you recreate the exact environment on any machine.

Commands

This command creates a new conda environment using the environment.yml file. It installs all listed packages and sets up Python version 3.9.

Terminal

conda env create -f environment.yml

Expected Output

Collecting package metadata (repodata.json): done

Solving environment: done



Downloading and Extracting Packages

...



Preparing transaction: done

Verifying transaction: done

Executing transaction: done



Environment 'my-ml-project' created successfully!

→

-f - Specifies the environment file to use for creation

This command switches your shell to use the newly created environment. Now, any Python or pip commands will use the packages from this environment.

Terminal

conda activate my-ml-project

Expected Output

(my-ml-project) $ 

This command shows all Python packages installed in the current environment via pip. It helps verify that pip packages like scikit-learn and matplotlib are installed.

Terminal

pip list

Expected Output

Package         Version

---------------- -------

matplotlib      3.6.2

pip             23.0.1

scikit-learn    1.1.3

setuptools      65.5.0

wheel           0.38.4

This command saves the current environment's full package list and versions to a file. You can share this file to recreate the environment exactly elsewhere.

Terminal

conda env export > environment_export.yml

Expected Output

No output (command runs silently)

This command exits the current conda environment and returns to the default system environment.

Terminal

conda deactivate

Expected Output

$ 

Key Concept

If you remember nothing else from this pattern, remember: conda manages environments and packages, while pip installs Python packages inside those environments.

Common Mistakes

Trying to install packages with pip outside an activated conda environment

Packages get installed globally or in the wrong environment, causing conflicts or missing packages in your project

Always activate the correct conda environment before running pip install

Editing the environment.yml file manually without proper indentation

YAML files are sensitive to indentation and format; errors cause environment creation to fail

Use a YAML editor or carefully maintain correct indentation and syntax

Not exporting the environment after installing new packages

The environment.yml file becomes outdated and others cannot recreate the exact setup

Run conda env export > environment_export.yml after changes to keep the file updated

Summary

Create a conda environment from an environment.yml file to manage Python and packages together.

Activate the environment to work inside it and use pip to install additional Python packages.

Export the environment to a file to share or recreate the setup exactly on other machines.