ML project structure in ML Python - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Identify the main purpose of the 'data' folder in an ML project

In a typical machine learning project, what is the main purpose of the data folder?

A To store raw and processed datasets used for training and testing models

B To save the final trained model files and checkpoints

C To keep scripts for data visualization and plotting results

D To hold configuration files for project settings

💡 Hint

Think about where you keep the information your model learns from.

The data folder is where all raw and processed datasets are stored. This includes training, validation, and test data. It is essential for the model to access this data during training and evaluation.

❓ Model Choice

intermediate

Choosing the right folder for model code

You have written Python code that defines your machine learning model architecture and training functions. Where should you place this code in a well-organized ML project?

A Inside a folder named <code>models</code> or <code>src/models</code>

B In the <code>data</code> folder alongside datasets

C In the root folder without any subfolder

D Inside the <code>notebooks</code> folder

💡 Hint

Think about where code related to model design logically belongs.

The models folder or a similar src/models path is the best place to keep model architecture and training code. This keeps code organized and separate from data and experiments.

❓ Metrics

advanced

Understanding evaluation metrics placement

Where in an ML project structure should you place code that calculates and logs evaluation metrics like accuracy or loss during model training?

A Directly inside the <code>data</code> folder

B Inside the <code>evaluation</code> or <code>metrics</code> folder or module

C In the <code>models</code> folder with the model architecture code

D In the <code>config</code> folder

💡 Hint

Think about where you keep code related to checking how well your model performs.

Evaluation metric code is best kept in a dedicated evaluation or metrics folder or module. This separates concerns and makes it easier to update or add new metrics.

🔧 Debug

advanced

Debugging misplaced files in ML project

You find that your ML project has training scripts inside the data folder and dataset files inside the models folder. What is the main problem caused by this structure?

A Training scripts will run faster due to proximity to data files

B The model will not train because data files are in the wrong folder

C It causes confusion and makes the project hard to maintain and understand

D There is no problem; folders can contain any files

💡 Hint

Think about how clear and easy it is to find things in a project.

Misplacing files causes confusion and makes the project difficult to maintain. Clear separation of data, code, and models helps teamwork and debugging.

❓ Hyperparameter

expert

Best practice for managing hyperparameters in ML projects

In a well-structured ML project, where should you store hyperparameters like learning rate, batch size, and number of epochs to ensure easy tuning and reproducibility?

A Hardcoded directly inside the training script code

B In the <code>models</code> folder alongside model architecture code

C Inside the <code>data</code> folder with datasets

D In a separate <code>config</code> file or folder, such as YAML or JSON files

💡 Hint

Think about how to change settings without editing code.

Storing hyperparameters in separate config files like YAML or JSON allows easy tuning without changing code. It also helps track experiments and reproduce results.

Practice

1. Which folder in an ML project should you use to store the original, untouched data files?

easy

A. Raw data folder

B. Processed data folder

C. Models folder

D. Scripts folder

Solution

1. Step 1: Understand the purpose of raw data folder

   The raw data folder is meant to keep the original data safe and unchanged.

2. Step 2: Differentiate from processed data folder

   The processed data folder contains cleaned or transformed data, not the original files.

3. Final Answer:

   Raw data folder -> Option A

4. Quick Check:

   Original data = Raw data folder [OK]

Hint: Original files go to raw data folder to keep them safe [OK]

Common Mistakes:

• Putting original data in processed folder
• Mixing models with data files
• Saving scripts in data folders

2. Which of the following is the correct way to organize code in an ML project?

easy

A. Put all code, notebooks, and models in one folder

B. Keep processed data and scripts mixed in one folder

C. Separate notebooks for exploration and scripts for training

D. Save models inside the raw data folder

Solution

1. Step 1: Identify the role of notebooks and scripts

   Notebooks are for exploring data and ideas; scripts are for running training and processing tasks.

2. Step 2: Understand folder separation

   Keeping notebooks and scripts separate helps keep the project organized and clear.

3. Final Answer:

   Separate notebooks for exploration and scripts for training -> Option C

4. Quick Check:

   Notebooks ≠ scripts folder [OK]

Hint: Use notebooks for exploring, scripts for running training [OK]

Common Mistakes:

• Mixing notebooks and scripts in one folder
• Saving models in data folders
• Putting all files in one folder

3. Given this ML project folder structure:

project/
├── data/
│   ├── raw/
│   └── processed/
├── notebooks/
├── scripts/
└── models/

Where should you save a trained model file after training?

medium

A. data/raw/

B. scripts/

C. notebooks/

D. models/

Solution

1. Step 1: Understand folder purposes

   The models folder is designed to store trained model files for reuse and deployment.

2. Step 2: Exclude other folders

   Data folders are for data files, notebooks for exploration, scripts for code execution, so models belong in models/.

3. Final Answer:

   models/ -> Option D

4. Quick Check:

   Trained model files go to models folder [OK]

Hint: Save trained models in the models folder [OK]

Common Mistakes:

• Saving models in data folders
• Putting models inside notebooks folder
• Confusing scripts folder with models

4. You find that your ML project has raw data and processed data mixed in the same folder. What is the best fix?

medium

A. Rename all files to indicate if they are raw or processed

B. Move raw data to a separate 'raw' folder and processed data to 'processed' folder

C. Delete the processed data and keep only raw data

D. Keep all data in one folder but add a README explaining the files

Solution

1. Step 1: Identify the problem

   Mixing raw and processed data risks overwriting or confusion about data versions.

2. Step 2: Organize data properly

   Separating raw and processed data into different folders keeps original data safe and processed data clear.

3. Final Answer:

   Move raw data to a separate 'raw' folder and processed data to 'processed' folder -> Option B

4. Quick Check:

   Separate raw and processed data folders [OK]

Hint: Keep raw and processed data in separate folders [OK]

Common Mistakes:

• Deleting processed data unnecessarily
• Only renaming files without moving
• Relying on README without folder separation

5. You want to share your ML project with a teammate. Which folder should you exclude to keep the project size small and avoid sharing large temporary files?

hard

A. outputs/

B. data/raw/

C. models/

D. notebooks/

Solution

1. Step 1: Identify large temporary files

   The outputs folder usually contains temporary results like logs, plots, or intermediate files that can be large and rebuilt.

2. Step 2: Decide what to share

   Models, raw data, and notebooks are important for reproducing work, so outputs can be excluded to save space.

3. Final Answer:

   outputs/ -> Option A

4. Quick Check:

   Exclude outputs folder to reduce project size [OK]

Hint: Exclude outputs folder to avoid sharing large temp files [OK]

Common Mistakes:

• Excluding raw data which is needed
• Removing models folder by mistake
• Not excluding any folder and sharing everything