PII detection and redaction in Prompt Engineering / GenAI - Full Explanation

Introduction

Imagine you have a big pile of documents with sensitive information like names and phone numbers. You want to share these documents but keep personal details private. This is where PII detection and redaction help by finding and hiding private data automatically.

Explanation

What is PII

PII stands for Personally Identifiable Information. It includes details like names, addresses, phone numbers, and social security numbers that can identify a person. Protecting PII is important to keep people's privacy safe.

PII is any information that can identify a specific person.

PII Detection

PII detection uses software to scan text or data and find pieces of information that are considered private. It looks for patterns like phone number formats or common name structures. This helps find sensitive data quickly without reading everything manually.

Detection finds private information automatically by recognizing patterns.

Redaction Process

Redaction means hiding or removing the detected PII from documents. This can be done by blacking out text, replacing it with symbols, or deleting it. Redaction ensures that when documents are shared, private details are not visible to others.

Redaction hides or removes private information to protect privacy.

Why It Matters

Sharing documents with PII exposed can lead to identity theft or privacy breaches. Using detection and redaction helps organizations follow laws and keep people's data safe. It also builds trust by showing respect for privacy.

Detecting and redacting PII prevents privacy risks and legal problems.

Real World Analogy

Think of sending a postcard but wanting to hide your home address. You use a marker to black out your address before mailing it. PII detection is like spotting your address on the postcard, and redaction is like using the marker to hide it.

What is PII → Your home address and phone number on a postcard

PII Detection → Noticing the address written on the postcard

Redaction Process → Using a marker to black out the address

Why It Matters → Protecting your privacy so strangers can’t find your home

Diagram

Diagram

┌───────────────┐     ┌───────────────┐     ┌───────────────┐
│   Document    │ --> │ PII Detection │ --> │  Redaction    │
└───────────────┘     └───────────────┘     └───────────────┘
                             │                     │
                             ▼                     ▼
                   Sensitive Info Found     Sensitive Info Hidden

This diagram shows the flow from a document through PII detection to redaction, highlighting how sensitive information is found and then hidden.

Key Facts

PII → Information that can identify a specific person, like name or phone number.

PII Detection → The process of automatically finding personal information in data.

Redaction → Hiding or removing sensitive information to protect privacy.

Privacy Breach → When private information is exposed without permission.

Data Protection Laws → Rules that require organizations to keep personal data safe.

Common Confusions

PII detection always finds every piece of personal data perfectly.

PII detection always finds every piece of personal data perfectly. PII detection tools can miss some information or flag non-sensitive data by mistake because patterns vary widely.

Redaction deletes the original data permanently from all systems.

Redaction deletes the original data permanently from all systems. Redaction hides data in shared documents but the original data may still exist in backups or databases.

Summary

PII detection helps find personal information automatically to protect privacy.

Redaction hides or removes sensitive data so it is not visible when sharing documents.

Together, they reduce risks of privacy breaches and help follow data protection laws.

Practice

1. What is the main purpose of PII detection in text data?

easy

A. To increase the size of the dataset

B. To improve the speed of text processing

C. To find personal information to protect privacy

D. To translate text into different languages

Solution

1. Step 1: Understand PII detection

   PII detection is about finding personal information like names, emails, or phone numbers in text.

2. Step 2: Identify the purpose

   The goal is to protect privacy by recognizing sensitive data that should not be shared openly.

3. Final Answer:

   To find personal information to protect privacy -> Option C

4. Quick Check:

   PII detection = find personal info [OK]

Hint: PII detection means finding personal info to keep it safe [OK]

Common Mistakes:

• Confusing PII detection with data translation
• Thinking it speeds up processing
• Believing it increases dataset size

2. Which of the following is the correct way to redact an email address in text?

easy

A. Replace the email with <EMAIL_REDACTED>

B. Delete the entire sentence containing the email

C. Change the email to a random number

D. Highlight the email in bold

Solution

1. Step 1: Understand redaction

   Redaction means hiding sensitive info by replacing it with a placeholder, not deleting or changing it randomly.

2. Step 2: Choose the correct method

   Replacing the email with a clear placeholder like <EMAIL_REDACTED> keeps the text readable and safe.

3. Final Answer:

   Replace the email with <EMAIL_REDACTED> -> Option A

4. Quick Check:

   Redaction = replace sensitive info with placeholder [OK]

Hint: Redact by replacing sensitive info with clear placeholders [OK]

Common Mistakes:

• Deleting whole sentences instead of redacting
• Replacing emails with unrelated data
• Highlighting instead of hiding

3. Given this Python code snippet for PII redaction:

import re
text = 'Contact me at [e-mail] or 123-456-7890.'
redacted = re.sub(r'\S+@\S+\.\S+', '<EMAIL_REDACTED>', text)
print(redacted)

What will be the output?

medium

A. Contact me at [e-mail] or 123-456-7890.

B. Contact me at [e-mail] or <EMAIL_REDACTED>.

C. Contact me at <EMAIL_REDACTED> or <EMAIL_REDACTED>.

D. Contact me at <EMAIL_REDACTED> or 123-456-7890.

Solution

1. Step 1: Understand the regex pattern

   The pattern '\S+@\S+\.\S+' matches email addresses (non-space chars @ non-space chars . non-space chars).

2. Step 2: Apply substitution

   The code replaces the email with '<EMAIL_REDACTED>' but leaves the phone number unchanged.

3. Final Answer:

   Contact me at <EMAIL_REDACTED> or 123-456-7890. -> Option D

4. Quick Check:

   Email replaced, phone unchanged = Contact me at <EMAIL_REDACTED> or 123-456-7890. [OK]

Hint: Regex replaces emails only, phone stays same [OK]

Common Mistakes:

• Thinking phone number is replaced
• Misreading regex pattern
• Assuming no replacement happens

4. You wrote this code to redact phone numbers:

import re
text = 'Call 555-1234 or 555-5678.'
redacted = re.sub(r'\d{3}-\d{4}', '<PHONE_REDACTED>', text)
print(redacted)

But the output is:
'Call 555-1234 or 555-5678.'
What is the likely error?

medium

A. The regex pattern is incorrect and does not match the phone numbers

B. The re.sub function is missing the text argument

C. The print statement is missing parentheses

D. The text variable is empty

Solution

1. Step 1: Check regex pattern against phone format

   The pattern '\d{3}-\d{4}' matches numbers like '555-1234', but the phone numbers might have different formats or extra spaces.

2. Step 2: Confirm if pattern matches text

   If the phone numbers have area codes or spaces, the pattern won't match, so no replacement occurs.

3. Final Answer:

   The regex pattern is incorrect and does not match the phone numbers -> Option A

4. Quick Check:

   Regex mismatch causes no replacement [OK]

Hint: Check regex matches exact phone format in text [OK]

Common Mistakes:

• Assuming re.sub syntax error
• Forgetting parentheses in print (Python 3+)
• Thinking text is empty without checking

5. You want to redact both emails and phone numbers in a text using Python. Which combined regex pattern correctly matches emails and US phone numbers like '123-456-7890'?

hard

A. r'\d{3}-\d{4}|\S+@\S+\.\S+'

B. r'\S+@\S+\.\S+|\d{3}-\d{3}-\d{4}'

C. r'\S+@\S+\.\S+\d{3}-\d{3}-\d{4}'

D. r'\S+@\S+\.\S+&\d{3}-\d{3}-\d{4}'

Solution

1. Step 1: Understand regex for emails and phones

   The email pattern '\S+@\S+\.\S+' matches emails; '\d{3}-\d{3}-\d{4}' matches US phone numbers like '123-456-7890'.

2. Step 2: Combine patterns with OR operator

   Using '|' between patterns matches either emails or phone numbers separately.

3. Step 3: Evaluate options

   r'\S+@\S+\.\S+|\d{3}-\d{3}-\d{4}' correctly uses '|' to combine patterns; r'\d{3}-\d{4}|\S+@\S+\.\S+' reverses order but still works; r'\S+@\S+\.\S+\d{3}-\d{3}-\d{4}' concatenates patterns (wrong); r'\S+@\S+\.\S+&\d{3}-\d{3}-\d{4}' uses '&' which is invalid in regex.

4. Final Answer:

   r'\S+@\S+\.\S+|\d{3}-\d{3}-\d{4}' -> Option B

5. Quick Check:

   Use '|' to combine regex patterns [OK]

Hint: Use '|' to combine email and phone regex patterns [OK]

Common Mistakes:

• Concatenating patterns without '|'
• Using invalid regex operators like '&'
• Mixing order but forgetting OR operator