Why advanced techniques handle complex data in ML Python - Experiment to Prove It

Experiment - Why advanced techniques handle complex data

Problem:You have a dataset with many features and complex patterns. A simple model like logistic regression is not capturing these patterns well.

Current Metrics:Training accuracy: 75%, Validation accuracy: 70%, Loss: 0.6

Issue:The model underfits the data because it is too simple to learn complex relationships.

Your Task

Improve the model to better capture complex patterns and increase validation accuracy to above 85%.

You can only change the model architecture and training parameters.

You cannot change the dataset or add new data.

Hint 1

Hint 2

Hint 3

Solution

ML Python

import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import Adam

# Create a complex synthetic dataset
X, y = make_classification(n_samples=1000, n_features=20, n_informative=15, n_redundant=5, random_state=42)

# Split data
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_val = scaler.transform(X_val)

# Build a deeper neural network model
model = Sequential([
    Dense(64, activation='relu', input_shape=(20,)),
    Dense(64, activation='relu'),
    Dense(1, activation='sigmoid')
])

# Compile model
model.compile(optimizer=Adam(learning_rate=0.001), loss='binary_crossentropy', metrics=['accuracy'])

# Train model
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_data=(X_val, y_val), verbose=0)

# Evaluate
train_acc = history.history['accuracy'][-1] * 100
val_acc = history.history['val_accuracy'][-1] * 100
train_loss = history.history['loss'][-1]
val_loss = history.history['val_loss'][-1]

print(f"Training accuracy: {train_acc:.2f}%")
print(f"Validation accuracy: {val_acc:.2f}%")
print(f"Training loss: {train_loss:.3f}")
print(f"Validation loss: {val_loss:.3f}")

Replaced simple logistic regression with a deeper neural network having two hidden layers.

Used ReLU activation functions to help model learn complex patterns.

Increased training epochs to 50 for better learning.

Results Interpretation

Before: Training accuracy 75%, Validation accuracy 70%, Loss 0.6

After: Training accuracy 92%, Validation accuracy 87%, Loss 0.3

Using advanced techniques like deeper neural networks with nonlinear activations helps the model learn complex data patterns better, improving accuracy and reducing underfitting.

Bonus Experiment

Try adding dropout layers to the neural network to reduce overfitting and see if validation accuracy improves further.

💡 Hint

Dropout randomly turns off some neurons during training, which helps the model generalize better.

Practice

1. Why do advanced machine learning techniques handle complex data better than simple methods?

easy

A. They require less data to train.

B. They always run faster than simple methods.

C. They ignore noisy data completely.

D. They can learn deeper patterns and relationships in the data.

Solution

1. Step 1: Understand the role of advanced techniques

   Advanced techniques like deep learning can find complex patterns that simple methods miss.

2. Step 2: Compare with simple methods

   Simple methods often fail on complex data because they cannot capture deep relationships.

3. Final Answer:

   They can learn deeper patterns and relationships in the data. -> Option D

4. Quick Check:

   Deeper pattern learning [OK]

Hint: Advanced methods find deep patterns, simple ones don't [OK]

Common Mistakes:

• Thinking advanced methods always run faster
• Believing advanced methods need less data
• Assuming advanced methods ignore noise

2. Which of the following is the correct way to import a deep learning model from TensorFlow in Python?

easy

A. import tensorflow as tf; model = keras.Sequential()

B. import tensorflow as tf; model = tf.deep.Sequential()

C. import tensorflow as tf; model = tf.keras.Sequential()

D. import tensorflow as tf; model = tf.keras.Model()

Solution

1. Step 1: Recall TensorFlow import syntax

   The standard way is to import tensorflow as tf and use tf.keras for models.

2. Step 2: Identify correct model creation

   tf.keras.Sequential() is the correct class for a simple deep learning model.

3. Final Answer:

   import tensorflow as tf; model = tf.keras.Sequential() -> Option C

4. Quick Check:

   tf.keras.Sequential() syntax [OK]

Hint: Use tf.keras.Sequential() to create models in TensorFlow [OK]

Common Mistakes:

• Using tf.deep instead of tf.keras
• Importing tensorflow as keras
• Using tf.keras.Model() for a sequential model

3. What will be the output shape of the following PyTorch model for input of shape (batch_size=10, channels=3, height=32, width=32)?

import torch
import torch.nn as nn
model = nn.Sequential(
  nn.Conv2d(3, 16, kernel_size=3, padding=1),
  nn.ReLU(),
  nn.MaxPool2d(2),
  nn.Conv2d(16, 32, kernel_size=3, padding=1),
  nn.ReLU(),
  nn.MaxPool2d(2)
)
input_tensor = torch.randn(10, 3, 32, 32)
output = model(input_tensor)
print(output.shape)

medium

A. (10, 32, 8, 8)

B. (10, 32, 16, 16)

C. (10, 16, 8, 8)

D. (10, 3, 32, 32)

Solution

1. Step 1: Calculate output after first Conv2d and MaxPool2d

   Conv2d keeps size 32x32 (padding=1, kernel=3), MaxPool2d halves it to 16x16 with 16 channels.

2. Step 2: Calculate output after second Conv2d and MaxPool2d

   Conv2d keeps size 16x16, MaxPool2d halves it to 8x8 with 32 channels.

3. Final Answer:

   (10, 32, 8, 8) -> Option A

4. Quick Check:

   Output shape = (batch, channels, height/4, width/4) [OK]

Hint: Each MaxPool2d halves height and width [OK]

Common Mistakes:

• Forgetting padding keeps size in Conv2d
• Not halving size after MaxPool2d
• Mixing up channel numbers

4. You have a neural network training code that runs but the accuracy stays very low. Which fix is most likely to improve the model's ability to handle complex data?

medium

A. Reduce the dataset size to speed up training.

B. Add more layers and neurons to the model.

C. Remove activation functions like ReLU.

D. Use only linear regression instead of neural networks.

Solution

1. Step 1: Understand model capacity and complexity

   More layers and neurons allow the model to learn complex patterns better.

2. Step 2: Evaluate other options

   Reducing data or removing activations reduces learning power; linear regression is too simple.

3. Final Answer:

   Add more layers and neurons to the model. -> Option B

4. Quick Check:

   Increasing model complexity [OK]

Hint: More layers = better complex pattern learning [OK]

Common Mistakes:

• Thinking less data helps accuracy
• Removing activation functions
• Replacing neural nets with linear regression

5. You want to classify images of cats and dogs using a dataset of 10,000 images. Which advanced technique is best suited to handle this complex image data and why?

hard

A. Use a convolutional neural network (CNN) because it learns spatial features automatically.

B. Use a decision tree because it handles images well without preprocessing.

C. Use k-nearest neighbors because it scales well with large image datasets.

D. Use linear regression because it is simple and fast.

Solution

1. Step 1: Identify the nature of image data

   Images have spatial patterns that CNNs can learn effectively through convolution layers.

2. Step 2: Compare other methods

   Decision trees and k-NN do not capture spatial features well; linear regression is unsuitable for classification.

3. Final Answer:

   Use a convolutional neural network (CNN) because it learns spatial features automatically. -> Option A

4. Quick Check:

   CNNs for images [OK]

Hint: CNNs automatically learn image features [OK]

Common Mistakes:

• Choosing decision trees for raw images
• Using k-NN without feature extraction
• Applying linear regression for classification