ML vs traditional programming in ML Python - Metrics Comparison

Metrics & Evaluation - ML vs traditional programming

Which metric matters for ML vs Traditional Programming and WHY

In traditional programming, correctness is measured by whether the program follows exact rules and produces the expected output every time. The main metric is accuracy of the program logic.

In machine learning, the program learns from data and makes predictions. Here, metrics like accuracy, precision, recall, and F1 score matter because predictions can be right or wrong with some uncertainty.

We use these metrics to understand how well the ML model generalizes beyond the training data, unlike traditional programs that follow fixed rules.

Confusion Matrix Example

For ML, a confusion matrix helps us see how many predictions were correct or wrong.

      Actual \ Predicted | Positive | Negative
      -------------------|----------|---------
      Positive           |    80    |   20    
      Negative           |    10    |   90    
    

This shows 80 true positives (TP), 20 false negatives (FN), 10 false positives (FP), and 90 true negatives (TN).

Precision vs Recall Tradeoff with Examples

In ML, precision and recall balance different goals:

• Precision: How many predicted positives are actually positive? Important when false alarms are costly, like spam filters (don't mark good emails as spam).
• Recall: How many actual positives did we find? Important when missing positives is costly, like cancer detection (don't miss any cancer cases).

Traditional programming does not have this tradeoff because it follows fixed rules.

Good vs Bad Metric Values for ML vs Traditional Programming

Traditional programming: 100% accuracy expected if rules are correct.

Machine learning: Good accuracy depends on the problem. For example, 95% accuracy might be good for image recognition but bad for rare disease detection where recall must be very high.

High precision but low recall means many positives are missed. High recall but low precision means many false alarms.

Common Pitfalls in Metrics

• Accuracy paradox: High accuracy can be misleading if data is imbalanced (e.g., 99% accuracy by always predicting the majority class).
• Data leakage: When test data leaks into training, metrics look better but model fails in real use.
• Overfitting: Model performs well on training data but poorly on new data, causing misleading metrics.
• Traditional programming: Bugs or logic errors cause wrong outputs, but metrics are usually binary (right or wrong).

Self Check

Your ML model has 98% accuracy but only 12% recall on fraud detection. Is it good for production?

Answer: No. Although accuracy is high, the model misses 88% of fraud cases (low recall). This is dangerous because catching fraud is critical. You need to improve recall even if accuracy drops.

Key Result

Traditional programming expects perfect accuracy; ML uses metrics like precision and recall to measure prediction quality and tradeoffs.

Practice

1. What is the main difference between traditional programming and machine learning?

easy

A. Traditional programming uses fixed rules; machine learning learns from data.

B. Traditional programming learns from data; machine learning uses fixed rules.

C. Both use fixed rules but in different languages.

D. Machine learning does not require any data.

Solution

1. Step 1: Understand traditional programming

   Traditional programming uses explicit instructions or fixed rules written by humans to solve problems.

2. Step 2: Understand machine learning

   Machine learning creates models that learn patterns from data examples instead of fixed rules.

3. Final Answer:

   Traditional programming uses fixed rules; machine learning learns from data. -> Option A

4. Quick Check:

   Difference = Fixed rules vs learning from data [OK]

Hint: Traditional = rules, ML = learns from examples [OK]

Common Mistakes:

• Confusing which method uses fixed rules
• Thinking ML does not need data
• Assuming both methods are the same

2. Which of the following is the correct way to describe traditional programming?

easy

A. It uses fixed rules written by programmers.

B. It predicts outcomes without any rules.

C. It learns rules from data automatically.

D. It requires no human input at all.

Solution

1. Step 1: Recall traditional programming definition

   Traditional programming relies on fixed rules explicitly coded by humans.

2. Step 2: Evaluate options

   Only It uses fixed rules written by programmers. correctly states this; others describe machine learning or incorrect ideas.

3. Final Answer:

   It uses fixed rules written by programmers. -> Option A

4. Quick Check:

   Traditional programming = fixed rules [OK]

Hint: Traditional = fixed rules by humans [OK]

Common Mistakes:

• Mixing up learning from data with fixed rules
• Thinking traditional programming predicts without rules
• Believing no human input is needed

3. Consider this code snippet for a traditional program and a machine learning model:

def predict_traditional(x):
    if x > 10:
        return 'High'
    else:
        return 'Low'

# Machine learning model trained on data
model = train_model(data)
prediction = model.predict(15)

What is the key difference in how these two make predictions?

medium

A. Both predict randomly without rules.

B. Both use fixed if-else rules to predict.

C. ML model uses if-else rules; traditional learns from data.

D. Traditional uses fixed if-else rules; ML model predicts based on learned patterns.

Solution

1. Step 1: Analyze traditional code

   The traditional function uses a fixed if-else rule to decide output based on input.

2. Step 2: Analyze ML model prediction

   The ML model predicts based on patterns learned from training data, not fixed rules.

3. Final Answer:

   Traditional uses fixed if-else rules; ML model predicts based on learned patterns. -> Option D

4. Quick Check:

   Traditional = rules, ML = learned patterns [OK]

Hint: Traditional = if-else rules; ML = learned from data [OK]

Common Mistakes:

• Assuming ML uses fixed if-else rules
• Thinking traditional learns from data
• Believing both predict randomly

4. A programmer wrote this code to classify emails as spam or not:

def classify_email(email):
    if 'free' in email:
        return 'spam'
    else:
        return 'not spam'

# Later, they try to use machine learning but wrote:
model = train_model(data)
result = model.predict('free money')

What is the main problem with this approach?

medium

A. The traditional code is better because it uses fixed rules.

B. The traditional code uses fixed rules; ML needs training data and proper model setup.

C. ML models do not work with text data.

D. The code will run without any issues.

Solution

1. Step 1: Identify traditional code limitation

   The traditional function uses a simple fixed rule that may miss many spam cases.

2. Step 2: Understand ML model needs

   ML requires proper training data and model setup before predicting; just calling predict without training causes errors or wrong results.

3. Final Answer:

   The traditional code uses fixed rules; ML needs training data and proper model setup. -> Option B

4. Quick Check:

   ML requires training, not just predict call [OK]

Hint: ML needs training data and setup before prediction [OK]

Common Mistakes:

• Thinking ML works without training
• Believing fixed rules are always better
• Ignoring model setup requirements

5. You have a complex problem where rules are hard to write, but you have lots of labeled data. Which approach is best and why? Options: A) Use traditional programming because rules are clearer. B) Use machine learning because it can learn from data patterns. C) Use traditional programming because it needs no data. D) Use neither because both require manual rules.

hard

A. Use traditional programming because rules are clearer.

B. Use traditional programming because it needs no data.

C. Use machine learning because it can learn from data patterns.

D. Use neither because both require manual rules.

Solution

1. Step 1: Analyze problem complexity and data availability

   When rules are hard to write but labeled data is available, machine learning can find patterns automatically.

2. Step 2: Compare approaches

   Traditional programming struggles with complex rules; ML excels with data-driven learning.

3. Final Answer:

   Use machine learning because it can learn from data patterns. -> Option C

4. Quick Check:

   Complex problem + data = machine learning [OK]

Hint: Complex rules + data = choose machine learning [OK]

Common Mistakes:

• Choosing traditional programming despite complex rules
• Ignoring data availability
• Thinking both require manual rules