ML project structure in ML Python

Introduction

Having a clear ML project structure helps keep your work organized and easy to follow. It makes teamwork and future updates simpler.

When starting a new machine learning project to keep files tidy.

When working with others so everyone understands where things are.

When you want to reuse parts of your project in the future.

When you need to track experiments and results clearly.

When preparing your project for deployment or sharing.

Syntax

ML Python

project_name/
  ├── data/
  │    ├── raw/
  │    ├── processed/
  ├── notebooks/
  ├── src/
  │    ├── data_processing.py
  │    ├── model.py
  │    ├── train.py
  ├── tests/
  ├── models/
  ├── outputs/
  │    ├── figures/
  │    ├── logs/
  ├── requirements.txt
  ├── README.md
  └── setup.py

The data/ folder stores your datasets, separated into raw and processed.

The src/ folder contains your code files like data processing and model training.

Examples

Keep original data in raw/ and cleaned or transformed data in processed/.

ML Python

project_name/
  ├── data/
  │    ├── raw/
  │    ├── processed/

Use notebooks/ for experiments and src/train.py for your training script.

ML Python

project_name/
  ├── notebooks/
  ├── src/
  │    ├── train.py

Save trained models in models/ and results like graphs and logs in outputs/.

ML Python

project_name/
  ├── models/
  ├── outputs/
  │    ├── figures/
  │    ├── logs/

Sample Model

This code creates the main folders for a machine learning project. It helps you start with a clean and organized setup.

ML Python

import os

# Create folders for ML project structure
folders = [
    'data/raw',
    'data/processed',
    'notebooks',
    'src',
    'tests',
    'models',
    'outputs/figures',
    'outputs/logs'
]

for folder in folders:
    os.makedirs(folder, exist_ok=True)

print('Folders created:')
for folder in folders:
    print(f'- {folder}')

Important Notes

Keep your code and data separate to avoid confusion.

Use README.md to explain your project and how to run it.

Track dependencies in requirements.txt for easy setup.

Summary

Organize your ML project with clear folders for data, code, models, and outputs.

Separate raw and processed data to keep original files safe.

Use notebooks for exploration and scripts for training and processing.