Bird
Raised Fist0

Implement Naive Bayes - Gaussian and Multinomial

CODINGintermediateInterview freq: medium
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
▶
Execution Trace
Step 1/5
Calculate mean and variance for Gaussian NB

Mean (μ) = 3.5, Variance (σ²) = 1.25 for feature 1

🎯 Understanding how Gaussian NB handles continuous data
mean = np.mean(feature)
variance = np.var(feature)
Compute likelihood for Gaussian NB

Likelihood for feature value 4.0 is 0.24

🎯 Calculating likelihood using Gaussian distribution
likelihood = (1 / np.sqrt(2 * np.pi * variance)) *
np.exp(-((value - mean) ** 2) / (2 * variance))
Count feature occurrences for Multinomial NB

Feature 'word1' appears 3 times in class 'A'

🎯 Counting occurrences for discrete data
count = feature_counts[class_label][feature]
Apply smoothing in Multinomial NB

Smoothed probability for 'word1' is 0.15

🎯 Understanding smoothing to handle zero probabilities
probability = (count + alpha) /
(total_count + alpha * num_features)
Calculate posterior probability

Posterior for class 'A' is 0.35

🎯 Combining likelihoods and priors to get posterior
posterior = prior * likelihood_product
📊
Diagram
Diagram
┌───────────────────────────────┐
│ Gaussian Naive Bayes         │
├───────────────────────────────┤
│ P(feature|class) =           │
│ 1/(sqrt(2πσ²)) * exp(-(x-μ)²/│
│ (2σ²))                       │
├───────────────────────────────┤
│ Multinomial Naive Bayes      │
├───────────────────────────────┤
│ P(feature|class) = (count+α)/│
│ (total_count+α*num_features) │
└───────────────────────────────┘
Labels
  • Gaussian Naive Bayes: Probability density function for continuous features
  • Multinomial Naive Bayes: Probability calculation for discrete features
  • P(feature|class): Conditional probability of a feature given a class
  • σ²: Variance of the feature in Gaussian Naive Bayes
  • α: Smoothing parameter in Multinomial Naive Bayes
🔑
Key Moments
▲
Q: Why is variance important in Gaussian Naive Bayes?
A: Variance determines the spread of the data and affects the likelihood calculation.
Q: How does smoothing affect Multinomial Naive Bayes?
A: Smoothing prevents zero probabilities by adding a constant to counts.
Q: What role does the prior play in Naive Bayes?
A: The prior represents the initial belief about class distribution before seeing data.
Trace Summary

This document provides a detailed execution trace of implementing Gaussian and Multinomial Naive Bayes classifiers. It covers the calculation of mean and variance for Gaussian Naive Bayes, likelihood computation, feature counting, and smoothing in Multinomial Naive Bayes. The diagram illustrates the probability formulas used in both classifiers. Key insights include understanding the importance of variance in Gaussian NB, the role of smoothing in Multinomial NB, and how priors influence the posterior probability. This trace helps in grasping the step-by-step process of Naive Bayes classification.

Primary Focus

✓ Naive Bayes implementation and execution trace

Practice

(1/5)
1. 
import numpy as np

def iou(box1, box2):
    x1 = max(box1[0], box2[0])
    y1 = max(box1[1], box2[1])
    x2 = min(box1[2], box2[2])
    y2 = min(box1[3], box2[3])
    intersection = max(0, x2 - x1) * max(0, y2 - y1)
    area1 = (box1[2] - box1[0]) * (box1[3] - box1[1])
    area2 = (box2[2] - box2[0]) * (box2[3] - box2[1])
    union = area1 + area2 - intersection
    return intersection / union

box1 = [1, 1, 3, 3]
box2 = [2, 2, 4, 4]
print(iou(box1, box2))
easy
A. 0.5
B. 0.25
C. 0.14285714285714285
D. 0.3333333333333333

Solution

  1. Step 1: Calculate intersection and union areas

    Intersection area is 1, union area is 7.

  2. Step 2: Compute IoU

    IoU = 1/7 = 0.14285714285714285.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Intersection over Union formula [OK]
Hint: Intersection over Union formula [OK]
Common Mistakes:
2. What is the time complexity of a single update step in Mini-Batch SGD with Momentum for a model with n parameters?
easy
A. O(n)
B. O(n log n)
C. O(n^2)
D. O(1)

Solution

  1. Step 1: Identify operations involved

    Each update step involves computing the gradient and updating the velocity and parameters, all of which are linear operations with respect to the number of parameters.

  2. Step 2: Determine the complexity

    Since each operation is linear, the overall complexity is O(n).

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Each parameter update is linear in complexity [OK]
Hint: Each parameter update is linear in complexity [OK]
Common Mistakes:
3. You implemented stratified K-Fold cross-validation but notice class imbalance in some folds. What is the most likely cause?
medium
A. Incorrect stratification parameter
B. Data shuffling before splitting
C. Incorrect number of splits
D. Too many folds for the dataset size

Solution

  1. Step 1: Identify stratification issue

    Stratification ensures class distribution is preserved in each fold.

  2. Step 2: Eliminate wrong options

    Incorrect number of splits and D relate to fold count, not class distribution. Data shuffling before splitting affects randomness, not stratification.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Check stratification parameter for class balance. [OK]
Hint: Check stratification parameter for class balance. [OK]
Common Mistakes:
4. You implemented gradient checking but the numerical and analytical gradients differ significantly. What is the most likely cause?
medium
A. The model is overfitting.
B. The step size for numerical gradient is too large.
C. The analytical gradient is incorrect.
D. The learning rate is too high.

Solution

  1. Step 1: Identify potential causes for gradient mismatch

    Significant differences often indicate errors in the analytical gradient.

  2. Step 2: Eliminate other options

    The step size for numerical gradient is too large. would cause small numerical errors, not significant differences. The learning rate is too high. and C are unrelated to gradient checking.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Mismatch often means analytical error. [OK]
Hint: Mismatch often means analytical error. [OK]
Common Mistakes:
5. In the following implementation of scaled dot-product attention, the output is incorrect. What is the likely cause? 
import torch
import torch.nn.functional as F

queries = torch.randn(2, 3)
keys = torch.randn(2, 3)
values = torch.randn(2, 3)

scores = torch.matmul(queries, keys.T) / torch.sqrt(torch.tensor(3.0))
attention_weights = F.softmax(scores, dim=0)
output = torch.matmul(attention_weights, values)
medium
A. Incorrect scaling factor used.
B. Softmax applied on the wrong dimension.
C. Values tensor has incorrect shape.
D. Scores tensor is not normalized.

Solution

  1. Step 1: Identify the issue

    Softmax should be applied on the last dimension, not the first.

  2. Step 2: Eliminate other causes

    Scaling factor and tensor shapes are correct; normalization is handled by softmax.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Softmax on last dim for attention [OK]
Hint: Softmax on last dim for attention [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard