Bird
Raised Fist0

Implement Logistic Regression - Binary Cross-Entropy, Sigmoid

CODINGbeginnerInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
โ–ถ
Execution Trace
Step 1/5
Input Data

X = [0.5, 1.5, 2.0] - input features for a single instance

๐ŸŽฏ Understanding input representation is crucial for feature engineering.
X = np.array([0.5, 1.5, 2.0])
Weights Initialization

W = [0.1, 0.2, 0.3] - initial weights set for the model

๐ŸŽฏ Initial weights can affect convergence speed.
W = np.array([0.1, 0.2, 0.3])
Linear Combination

z = 0.5*0.1 + 1.5*0.2 + 2.0*0.3 = 0.95

๐ŸŽฏ Linear combination is the core of logistic regression before applying sigmoid.
z = np.dot(W, X)
Sigmoid Activation

ฯƒ(z) = 1/(1+e^-0.95) โ‰ˆ 0.721

๐ŸŽฏ Sigmoid function maps any real-valued number into (0, 1).
y_pred = 1 / (1 + np.exp(-z))
Loss Calculation

L = -1*log(0.721) - (1-1)*log(1-0.721) โ‰ˆ 0.327

๐ŸŽฏ Binary cross-entropy loss measures the performance of the model.
loss = -y_true * np.log(y_pred) - (1 - y_true) * np.log(1 - y_pred)
๐Ÿ“Š
Diagram
Diagram
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ Logistic Regression Process  โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Input Data โ”‚ X = [x1, x2, x3]โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Weights    โ”‚ W = [w1, w2, w3]โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Sigmoid    โ”‚ ฯƒ(z) = 1/(1+e^-z)โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Prediction โ”‚ ลท = ฯƒ(WยทX + b)  โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Loss       โ”‚ L = -ylog(ลท)-(1-y)log(1-ลท) โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
Labels
  • Input Data: Features used for prediction
  • Weights: Parameters learned during training
  • Sigmoid: Activation function for binary classification
  • Prediction: Output of the model
  • Loss: Binary cross-entropy loss function
๐Ÿ”‘
Key Moments
โ–ฒ
Q: Why is the sigmoid function used in logistic regression?
A: The sigmoid function maps predictions to a probability between 0 and 1, suitable for binary classification.
Q: How does the choice of initial weights affect the training process?
A: Initial weights can influence the convergence speed and the likelihood of reaching a global minimum.
Q: What role does the binary cross-entropy loss play in logistic regression?
A: It quantifies the difference between predicted probabilities and actual labels, guiding weight updates.
Trace Summary

This diagram and trace illustrate the logistic regression process, focusing on input data, weight initialization, linear combination, sigmoid activation, and loss calculation. The sigmoid function transforms linear combinations into probabilities, while binary cross-entropy loss evaluates prediction accuracy. Understanding these steps is crucial for implementing logistic regression effectively.

Primary Focus

โœ“ Implementing logistic regression using binary cross-entropy and sigmoid function

Practice

(1/5)
1. 
import numpy as np
from sklearn.metrics import confusion_matrix

y_true = [0, 1, 0, 1, 0, 1, 1]
y_pred = [0, 0, 0, 1, 1, 1, 1]

cm = confusion_matrix(y_true, y_pred)
print(cm)
What does the code output?
easy
A. [[2, 1], [1, 3]]
B. [[3, 1], [1, 2]]
C. [[2, 2], [1, 2]]
D. [[3, 0], [2, 2]]

Solution

  1. Step 1: Understand confusion matrix structure

    The confusion matrix is structured as [[TN, FP], [FN, TP]].

  2. Step 2: Calculate values

    TN=2 (true negatives), FP=1 (false positives), FN=1 (false negatives), TP=3 (true positives).

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Confusion matrix is [[TN, FP], [FN, TP]]. [OK]
Hint: Confusion matrix is [[TN, FP], [FN, TP]]. [OK]
Common Mistakes:
2. What is the time complexity of computing the ROC curve using the sklearn library?
easy
A. O(n log n)
B. O(n^2)
C. O(n)
D. O(log n)

Solution

  1. Step 1: Understand ROC computation

    ROC curve computation involves sorting the scores, which is O(n log n).

  2. Step 2: Eliminate wrong options

    O(n^2) is incorrect as it suggests quadratic complexity, O(n) suggests linear complexity which ignores sorting, and O(log n) is too low.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    ROC curve involves sorting, hence O(n log n) [OK]
Hint: ROC curve involves sorting, hence O(n log n) [OK]
Common Mistakes:
3. A Gaussian Naive Bayes model is giving poor performance on a dataset. Which of the following is the most likely cause?
medium
A. The model has too many parameters.
B. The features are not normally distributed.
C. The dataset is too small.
D. The features are highly correlated.

Solution

  1. Step 1: Identify Gaussian Naive Bayes assumptions

    Gaussian Naive Bayes assumes that features are normally distributed.

  2. Step 2: Evaluate each option

    The features are not normally distributed. is correct because non-normal distribution violates the model's assumptions. The model has too many parameters. is incorrect as GaussianNB is simple. The dataset is too small. is less likely as small datasets affect all models. The features are highly correlated. is incorrect as GaussianNB can handle correlated features.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    GaussianNB assumes normal distribution. [OK]
Hint: GaussianNB assumes normal distribution. [OK]
Common Mistakes:
4. In the following implementation of scaled dot-product attention, the output is incorrect. What is the likely cause? 
import torch
import torch.nn.functional as F

queries = torch.randn(2, 3)
keys = torch.randn(2, 3)
values = torch.randn(2, 3)

scores = torch.matmul(queries, keys.T) / torch.sqrt(torch.tensor(3.0))
attention_weights = F.softmax(scores, dim=0)
output = torch.matmul(attention_weights, values)
medium
A. Incorrect scaling factor used.
B. Softmax applied on the wrong dimension.
C. Values tensor has incorrect shape.
D. Scores tensor is not normalized.

Solution

  1. Step 1: Identify the issue

    Softmax should be applied on the last dimension, not the first.

  2. Step 2: Eliminate other causes

    Scaling factor and tensor shapes are correct; normalization is handled by softmax.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Softmax on last dim for attention [OK]
Hint: Softmax on last dim for attention [OK]
Common Mistakes:
5. How would you modify the scaled dot-product attention mechanism to handle variable-length sequences efficiently?
medium
A. Use padding and mask the padded values.
B. Use a fixed-size window for all sequences.
C. Truncate all sequences to the same length.
D. Ignore sequences longer than a threshold.

Solution

  1. Step 1: Identify the challenge

    Variable-length sequences require handling of different sequence lengths.

  2. Step 2: Choose the correct approach

    Padding with masking is efficient and maintains sequence integrity.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Pad and mask for variable lengths [OK]
Hint: Pad and mask for variable lengths [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard