Data pipelines with DVC in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

DVC Pipeline Stage Output

You run the command dvc run -n preprocess -d data/raw.csv -o data/processed.csv python preprocess.py. What will be the output of dvc pipeline show immediately after?

A preprocess

B No stages found

C preprocess -> train

D train

💡 Hint

Think about what stages are created after running dvc run.

The command creates a stage named 'preprocess'. Running 'dvc pipeline show' lists existing stages. Since only 'preprocess' is created, it shows just that.

❓ Configuration

intermediate

Correct DVC Stage Definition in dvc.yaml

Which of the following dvc.yaml stage definitions correctly specifies a stage named 'train' that depends on 'data/processed.csv' and runs 'python train.py' producing 'model.pkl'?

A

stages:
  train:
    cmd: python train.py
    deps:
      - data/processed.csv
    outs:
      - model.pkl

B

stages:
  train:
    cmd: python train.py
    outs:
      - data/processed.csv
    deps:
      - model.pkl

C

stages:
  train:
    cmd: python train.py
    deps:
      - model.pkl
    outs:
      - data/processed.csv

D

stages:
  train:
    cmd: python train.py
    deps:
      - model.pkl
    outs:
      - model.pkl

💡 Hint

Remember dependencies are inputs, outputs are results.

Option A correctly lists 'data/processed.csv' as a dependency and 'model.pkl' as an output. The command is correct.

❓ Troubleshoot

advanced

DVC Pipeline Reproduction Issue

You modified 'preprocess.py' but running dvc repro does not rerun the 'preprocess' stage. What is the most likely cause?

A The output file 'data/processed.csv' was deleted manually.

B The 'preprocess.py' file is not listed as a dependency in the dvc.yaml stage.

C The DVC cache is full and needs cleaning.

D The 'dvc.lock' file is missing.

💡 Hint

DVC tracks changes only in declared dependencies.

If 'preprocess.py' is not declared as a dependency, DVC does not detect changes in it and skips rerunning the stage.

🔀 Workflow

advanced

DVC Pipeline Stage Execution Order

Given a pipeline with stages: 'download' -> 'preprocess' -> 'train', which command will reproduce only the 'train' stage and all its dependencies?

A dvc repro

B dvc repro preprocess

C dvc repro download

D dvc repro train

💡 Hint

Reproducing a stage also reproduces its dependencies.

Running 'dvc repro train' will reproduce 'train' and all stages it depends on, including 'preprocess' and 'download'.

✅ Best Practice

expert

Best Practice for Large Data Files in DVC Pipelines

You have very large raw data files that rarely change but are needed for multiple pipeline runs. What is the best practice to manage these files with DVC?

A Commit the large raw data files directly to Git for easy access.

B Exclude the large raw data files from DVC and Git, and manually copy them before each run.

C Track the large raw data files with DVC and store them in remote storage, avoiding committing them to Git.

D Compress the large raw data files and commit the compressed files to Git.

💡 Hint

Think about versioning large files efficiently without bloating Git.

DVC is designed to track large files efficiently by storing them in remote storage and keeping lightweight pointers in Git. This avoids bloating Git repositories.