Privacy considerations in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Privacy considerations

Which metric matters for Privacy considerations and WHY

Privacy in machine learning is about protecting personal data. Metrics here focus on how well the model or system keeps data safe. Common metrics include differential privacy guarantees that measure how much information about an individual can leak from the model. Another key metric is membership inference attack success rate, which shows how easily an attacker can tell if a person's data was used to train the model. Lower values mean better privacy.

Confusion matrix or equivalent visualization

For privacy attacks like membership inference, a confusion matrix shows how well an attacker guesses if data was in training:

      | Actual In Training | Actual Not In Training |
      |--------------------|-----------------------|
      | True Positive (TP)  | False Positive (FP)    |
      | False Negative (FN) | True Negative (TN)     |
    

Privacy risk is higher if TP and FP are high, meaning the attacker guesses membership correctly or wrongly often. A good privacy model has low TP and FP for the attacker.

Precision vs Recall tradeoff with concrete examples

In privacy attacks, precision means how often the attacker's positive guess is correct (true membership). Recall means how many actual members the attacker finds.

High precision, low recall: Attacker is sure when guessing membership but misses many members. Privacy is better.

High recall, low precision: Attacker finds many members but also guesses wrongly often. Privacy risk is still high.

For privacy, we want both precision and recall of attacks to be low, meaning the attacker cannot reliably find training data.

What "good" vs "bad" metric values look like for Privacy considerations

• Good: Differential privacy epsilon close to 0 (strong privacy), attacker precision and recall near random guessing (e.g., 0.5), low membership inference attack success.
• Bad: High epsilon (weak privacy), attacker precision and recall close to 1 (attacker can easily identify training data), high leakage of sensitive info.

Metrics pitfalls in Privacy considerations

• Ignoring privacy metrics: Focusing only on accuracy can hide privacy risks.
• Data leakage: If training data leaks into test sets, privacy metrics may be misleading.
• Overfitting: Models that memorize training data increase privacy risk but may show good accuracy.
• Misinterpreting epsilon: Not understanding that smaller epsilon means better privacy.

Self-check: Your model has 98% accuracy but 12% recall on fraud. Is it good?

This question is about fraud detection, not privacy, but it shows why metrics matter. High accuracy can be misleading if fraud cases are rare. A 12% recall means the model finds only 12% of frauds, which is poor. For privacy, similarly, a model can be accurate but leak private data. Always check privacy metrics, not just accuracy.

Key Result

Privacy metrics focus on limiting data leakage, with low membership inference attack success and strong differential privacy guarantees indicating good privacy.

Practice

1. What is the main reason to consider privacy in machine learning?

easy

A. To protect personal data from being exposed

B. To make the model run faster

C. To increase the size of the dataset

D. To improve the color of data visualizations

Solution

1. Step 1: Understand the role of privacy

   Privacy in machine learning is about keeping personal data safe and secure.

2. Step 2: Identify the main goal

   The main goal is to protect personal data from being exposed or misused.

3. Final Answer:

   To protect personal data from being exposed -> Option A

4. Quick Check:

   Privacy = Protect personal data [OK]

Hint: Privacy means keeping personal data safe [OK]

Common Mistakes:

• Thinking privacy speeds up models
• Confusing privacy with data size
• Mixing privacy with visualization features

2. Which of the following is a correct way to protect privacy before training a model?

easy

A. Include all personal details in the dataset

B. Remove or mask personal identifiers from data

C. Share the raw data publicly

D. Use the same password for all data files

Solution

1. Step 1: Review data preparation for privacy

   Removing or masking personal identifiers helps hide sensitive information.

2. Step 2: Check options for privacy protection

   Including personal details or sharing raw data exposes privacy; using weak passwords is unsafe.

3. Final Answer:

   Remove or mask personal identifiers from data -> Option B

4. Quick Check:

   Mask personal data = Protect privacy [OK]

Hint: Mask or remove personal info before training [OK]

Common Mistakes:

• Keeping all personal details
• Sharing raw data openly
• Ignoring data security practices

3. Consider this Python code snippet for data anonymization:

def anonymize(data):
    return [{k: (v if k != 'name' else 'REDACTED') for k, v in record.items()} for record in data]

sample_data = [{'name': 'Alice', 'age': 30}, {'name': 'Bob', 'age': 25}]
print(anonymize(sample_data))

What is the output?

medium

A. [{'name': 'REDACTED', 'age': 30}, {'name': 'REDACTED', 'age': 25}]

B. SyntaxError

C. [{'age': 30}, {'age': 25}]

D. [{'name': 'Alice', 'age': 30}, {'name': 'Bob', 'age': 25}]

Solution

1. Step 1: Understand the anonymize function

   The function replaces the value of the 'name' key with 'REDACTED' for each record.

2. Step 2: Apply the function to sample_data

   Names 'Alice' and 'Bob' become 'REDACTED', ages remain unchanged.

3. Final Answer:

   [{'name': 'REDACTED', 'age': 30}, {'name': 'REDACTED', 'age': 25}] -> Option A

4. Quick Check:

   Names replaced with REDACTED = [{'name': 'REDACTED', 'age': 30}, {'name': 'REDACTED', 'age': 25}] [OK]

Hint: Check key replacement in dict comprehension [OK]

Common Mistakes:

• Thinking names stay the same
• Removing the 'name' key instead of replacing
• Confusing syntax causing errors

4. The following code is intended to remove personal emails from a dataset but has a bug:

def remove_emails(data):
    for record in data:
        if 'email' in record:
            record.pop('email')
    return data

sample = [{'name': 'John', 'email': '[e-mail]'}, {'name': 'Jane'}]
print(remove_emails(sample))

What is the bug and how to fix it?

medium

A. Loop should iterate over a copy of data to avoid modification errors

B. pop() should be pop('email', None) to avoid errors

C. No bug; code works correctly

D. Function should return a new list without emails instead of modifying in place

Solution

1. Step 1: Analyze the pop usage

   pop('email') removes the 'email' key if it exists; no error if checked first.

2. Step 2: Check the loop and data modification

   Loop modifies records in place safely; no iteration over changing list structure.

3. Final Answer:

   No bug; code works correctly -> Option C

4. Quick Check:

   pop after key check = safe removal [OK]

Hint: Check if key exists before pop to avoid errors [OK]

Common Mistakes:

• Assuming pop causes error without key check
• Thinking list copy needed for dict modification
• Believing function must create new list always

5. You want to train a machine learning model on health data but must ensure privacy. Which combined approach best protects privacy while allowing model training?

hard

A. Publish all data publicly but only share model predictions

B. Use raw data with no changes and share model openly

C. Encrypt data but train model without removing identifiers

D. Remove personal identifiers and apply differential privacy during training

Solution

1. Step 1: Identify privacy protection methods

   Removing personal identifiers hides direct personal info; differential privacy adds noise to protect indirect info.

2. Step 2: Evaluate options for privacy and training

   Using raw data or publishing data openly risks privacy; encryption alone doesn't anonymize data for training.

3. Final Answer:

   Remove personal identifiers and apply differential privacy during training -> Option D

4. Quick Check:

   Combine anonymization + differential privacy = Remove personal identifiers and apply differential privacy during training [OK]

Hint: Combine data cleaning with privacy techniques for safety [OK]

Common Mistakes:

• Using raw data without protection
• Relying only on encryption without anonymization
• Publishing sensitive data openly