Logistic regression in ML Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the main purpose of logistic regression?

Logistic regression is used to predict the probability of a binary outcome, like yes/no or true/false, based on input features.

beginner

What function does logistic regression use to map predictions to probabilities?

It uses the sigmoid function, which takes any number and squashes it into a value between 0 and 1, representing a probability.

intermediate

Explain the difference between linear regression and logistic regression.

Linear regression predicts continuous values, while logistic regression predicts probabilities for categories (usually two classes). Logistic regression uses the sigmoid function to convert linear outputs into probabilities.

intermediate

What metric is commonly used to measure the performance of a logistic regression model?

Accuracy is common, but also log loss (cross-entropy loss) is used because it measures how close the predicted probabilities are to the actual labels.

advanced

Why do we use the log-odds (logit) in logistic regression?

The log-odds transform the probability into a continuous value that can be modeled as a linear combination of features, making it easier to fit with linear methods.

What output does logistic regression produce?

A A probability between 0 and 1

B A continuous number without limits

C A categorical label directly

D A cluster assignment

Logistic regression outputs probabilities between 0 and 1 using the sigmoid function.

Which function is used in logistic regression to convert linear predictions to probabilities?

A Sigmoid

B Softmax

C Tanh

D ReLU

The sigmoid function maps any real number to a value between 0 and 1, suitable for binary classification.

Which loss function is typically minimized in logistic regression training?

A Mean squared error

B Cross-entropy loss

C Hinge loss

D Absolute error

Cross-entropy loss measures the difference between predicted probabilities and actual labels, ideal for classification.

Logistic regression is best suited for which type of problem?

A Predicting house prices

B Reducing data dimensions

C Clustering customers

D Classifying emails as spam or not spam

Logistic regression is used for binary classification tasks like spam detection.

What does the term 'log-odds' refer to in logistic regression?

A The odds of an event squared

B The logarithm of the probability

C The logarithm of the odds ratio of an event happening

D The inverse of the probability

Log-odds is the log of the ratio of the probability of an event happening to it not happening.

Describe how logistic regression predicts the class of a new data point.

Think about how numbers become probabilities and then classes.

Explain why logistic regression uses the sigmoid function instead of a linear function for classification.

Consider what a probability must look like.

Practice

1. What is the main purpose of logistic regression in machine learning?

easy

A. To cluster data points into groups

B. To predict binary outcomes like yes/no or true/false

C. To reduce the number of features in data

D. To generate random data samples

Solution

1. Step 1: Understand logistic regression goal

   Logistic regression is used to predict categories with two possible outcomes, such as yes/no or true/false.

2. Step 2: Compare with other options

   Clustering groups data (not logistic regression), feature reduction simplifies data, and random sampling is unrelated.

3. Final Answer:

   To predict binary outcomes like yes/no or true/false -> Option B

4. Quick Check:

   Logistic regression = binary prediction [OK]

Hint: Logistic regression predicts yes/no answers fast [OK]

Common Mistakes:

• Confusing logistic regression with clustering
• Thinking it reduces features
• Assuming it generates data

2. Which of the following is the correct way to import logistic regression from scikit-learn in Python?

easy

A. import logistic_regression from sklearn.linear_model

B. import LogisticRegression from sklearn.linear_model

C. from sklearn import LogisticRegression

D. from sklearn.linear_model import LogisticRegression

Solution

1. Step 1: Recall correct import syntax

   In Python, the correct import uses 'from module import class' format, so 'from sklearn.linear_model import LogisticRegression' is correct.

2. Step 2: Identify incorrect syntax

   Options B, C, and D use wrong order or wrong names causing syntax errors.

3. Final Answer:

   from sklearn.linear_model import LogisticRegression -> Option D

4. Quick Check:

   Correct import = from sklearn.linear_model import LogisticRegression [OK]

Hint: Use 'from module import class' for sklearn imports [OK]

Common Mistakes:

• Using wrong import order
• Wrong class or module names
• Using lowercase instead of correct class name

3. What will be the output of this Python code snippet?

from sklearn.linear_model import LogisticRegression
import numpy as np

X = np.array([[0], [1], [2], [3]])
y = np.array([0, 0, 1, 1])
model = LogisticRegression()
model.fit(X, y)
pred = model.predict([[1.5]])
print(pred)

medium

A. [1]

B. [0]

C. [2]

D. Error

Solution

1. Step 1: Understand training data and labels

   X has values 0,1 labeled 0 and 2,3 labeled 1. The model learns to separate near 1.5 between classes 0 and 1.

2. Step 2: Predict for input 1.5

   Since 1.5 is closer to class 1 range, logistic regression predicts class 1.

3. Final Answer:

   [1] -> Option A

4. Quick Check:

   Input 1.5 predicts class 1 [OK]

Hint: Predict near boundary to guess class [OK]

Common Mistakes:

• Assuming prediction is 0 for 1.5
• Expecting error due to input shape
• Confusing output format

4. Identify the error in this logistic regression code snippet:

from sklearn.linear_model import LogisticRegression
X = [[0], [1], [2], [3]]
y = [0, 0, 1, 1]
model = LogisticRegression()
model.fit(X, y)
pred = model.predict([1.5])
print(pred)

medium

A. Input to predict should be 2D array, not 1D

B. Labels y should be strings, not integers

C. Model must be initialized with parameters

D. fit method requires numpy arrays, not lists

Solution

1. Step 1: Check input shape for prediction

   Model expects 2D array for predict input, but [1.5] is 1D. It should be [[1.5]].

2. Step 2: Verify other parts

   Labels as integers are fine, model can be default, and fit accepts lists converted internally.

3. Final Answer:

   Input to predict should be 2D array, not 1D -> Option A

4. Quick Check:

   Predict input shape = 2D array [OK]

Hint: Always use double brackets for single sample prediction [OK]

Common Mistakes:

• Passing 1D array to predict
• Thinking labels must be strings
• Assuming model needs parameters to initialize

5. You have a dataset with features that are not scaled and vary widely in range. What is the best practice before applying logistic regression to improve model performance?

hard

A. Use logistic regression without any preprocessing

B. Remove half of the features randomly

C. Scale or normalize features to similar ranges

D. Increase the number of training samples without scaling

Solution

1. Step 1: Understand feature scaling importance

   Logistic regression uses weights sensitive to feature scale; large differences can cause poor training.

2. Step 2: Choose best preprocessing

   Scaling or normalizing features to similar ranges helps the model learn better and converge faster.

3. Final Answer:

   Scale or normalize features to similar ranges -> Option C

4. Quick Check:

   Feature scaling improves logistic regression [OK]

Hint: Always scale features before logistic regression [OK]

Common Mistakes:

• Ignoring feature scaling
• Randomly removing features
• Assuming more data fixes scaling issues