First neural network in TensorFlow

Introduction

A neural network helps a computer learn patterns from data to make decisions or predictions.

When you want to recognize handwritten numbers.

When you want to predict if an email is spam or not.

When you want to classify images into categories like cats or dogs.

When you want to predict house prices based on features.

When you want to understand simple relationships in data.

Syntax

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models

model = models.Sequential([
    layers.Dense(units=10, activation='relu', input_shape=(784,)),
    layers.Dense(units=1, activation='sigmoid')
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)

Sequential means layers are stacked one after another.

Dense is a fully connected layer where each input connects to each output.

Examples

A small network with 3 inputs, one hidden layer with 5 neurons, and one output neuron.

TensorFlow

model = models.Sequential([
    layers.Dense(5, activation='relu', input_shape=(3,)),
    layers.Dense(1, activation='sigmoid')
])

Compile model for regression using mean squared error loss and stochastic gradient descent optimizer.

TensorFlow

model.compile(optimizer='sgd', loss='mean_squared_error', metrics=['mse'])

Train the model for 10 rounds with batches of 32 samples.

TensorFlow

model.fit(x_train, y_train, epochs=10, batch_size=32)

Sample Model

This program creates a tiny dataset where the output is 1 if the sum of inputs is greater than or equal to 1, else 0. It builds a small neural network with one hidden layer, trains it, and shows predictions and accuracy.

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models
import numpy as np

# Create simple data: inputs are 2 numbers, output is 1 if sum >= 1 else 0
x_train = np.array([[0,0],[0,1],[1,0],[1,1]], dtype=float)
y_train = np.array([0,1,1,1], dtype=float)

# Build model
model = models.Sequential([
    layers.Dense(4, activation='relu', input_shape=(2,)),
    layers.Dense(1, activation='sigmoid')
])

# Compile model
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Train model
history = model.fit(x_train, y_train, epochs=20, verbose=0)

# Predict on training data
predictions = model.predict(x_train)

# Print predictions rounded to 2 decimals
for i, pred in enumerate(predictions):
    print(f"Input: {x_train[i]}, Prediction: {pred[0]:.2f}, Actual: {y_train[i]}")

# Print final accuracy
final_acc = history.history['accuracy'][-1]
print(f"Final training accuracy: {final_acc:.2f}")

Important Notes

Use relu activation to help the network learn complex patterns.

sigmoid activation is good for outputs between 0 and 1, like yes/no.

Training for more epochs usually improves accuracy but watch out for overfitting.

Summary

A neural network learns from data by adjusting connections between layers.

Start with a simple model: input layer, one hidden layer, and output layer.

Use compile to set how the model learns and fit to train it.

Practice

1. What is the main purpose of the compile method in a TensorFlow neural network model?

easy

A. To set the optimizer, loss function, and metrics for training

B. To add layers to the model

C. To train the model on data

D. To make predictions on new data

Solution

1. Step 1: Understand the role of compile

   The compile method prepares the model for training by specifying how it learns, including the optimizer, loss function, and metrics.

2. Step 2: Differentiate from other methods

   Adding layers is done before compiling, training is done with fit, and predictions use predict.

3. Final Answer:

   To set the optimizer, loss function, and metrics for training -> Option A

4. Quick Check:

   compile sets training details = A [OK]

Hint: Compile sets how the model learns before training [OK]

Common Mistakes:

• Confusing compile with fit (training)
• Thinking compile adds layers
• Mixing compile with prediction

2. Which of the following is the correct way to add a dense hidden layer with 10 neurons and ReLU activation in TensorFlow?

easy

A. model.add(tf.keras.Dense(10, activation='relu'))

B. model.add(Dense(activation='relu', 10))

C. model.add(tf.keras.layers.Dense(10, activation='relu'))

D. model.add(tf.layers.Dense(activation='relu', units=10))

Solution

1. Step 1: Recall correct TensorFlow syntax for adding layers

   The correct way is to use tf.keras.layers.Dense with units first, then activation as a named argument.

2. Step 2: Check each option

   model.add(tf.keras.layers.Dense(10, activation='relu')) matches the correct syntax. model.add(Dense(activation='relu', 10)) has wrong argument order. model.add(tf.layers.Dense(activation='relu', units=10)) uses deprecated tf.layers. model.add(tf.keras.Dense(10, activation='relu')) misses layers in the path.

3. Final Answer:

   model.add(tf.keras.layers.Dense(10, activation='relu')) -> Option C

4. Quick Check:

   Correct layer syntax = D [OK]

Hint: Use tf.keras.layers.Dense(units, activation='relu') [OK]

Common Mistakes:

• Wrong argument order in Dense layer
• Using deprecated tf.layers instead of tf.keras.layers
• Missing 'layers' in the import path

3. What will be the output shape of the model after adding these layers?

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(5, input_shape=(3,), activation='relu'))
model.add(tf.keras.layers.Dense(2, activation='softmax'))
print(model.output_shape)

medium

A. (None, 5)

B. (None, 2)

C. (None, 3)

D. (3, 2)

Solution

1. Step 1: Understand input and output shapes

   The input shape is (3,), first layer outputs 5 units, second layer outputs 2 units.

2. Step 2: Determine final output shape

   The model output shape is (None, 2) where None is batch size, 2 is output units.

3. Final Answer:

   (None, 2) -> Option B

4. Quick Check:

   Output units = 2 means shape (None, 2) [OK]

Hint: Output shape matches last layer units with batch size None [OK]

Common Mistakes:

• Confusing input shape with output shape
• Ignoring batch size dimension None
• Mixing layer units and input dimensions

4. Identify the error in this code snippet for creating a simple neural network:

model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(10, activation='relu'))
model.compile(optimizer='adam', loss='mse')
model.summary()
model.fit(x_train, y_train, epochs=5)

medium

A. Optimizer 'adam' is not supported

B. Loss function 'mse' is invalid

C. fit method requires batch_size argument

D. Missing input shape in the first layer

Solution

1. Step 1: Check layer definition

   The first Dense layer lacks an input shape, which is required for the model to know input dimensions.

2. Step 2: Verify other parts

   Loss 'mse' and optimizer 'adam' are valid. Batch size is optional in fit.

3. Final Answer:

   Missing input shape in the first layer -> Option D

4. Quick Check:

   Input shape needed in first layer = C [OK]

Hint: Always specify input shape in first layer [OK]

Common Mistakes:

• Skipping input_shape in first layer
• Thinking batch_size is mandatory in fit
• Confusing loss and optimizer names

5. You want to build a neural network to classify images into 3 categories. Which model setup is best?

model = tf.keras.Sequential([
  tf.keras.layers.Flatten(input_shape=(28,28)),
  tf.keras.layers.Dense(64, activation='relu'),
  tf.keras.layers.Dense(3, activation='softmax')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

hard

A. Correct setup for multi-class classification

B. Use sigmoid activation in last layer instead of softmax

C. Use mean squared error loss for classification

D. Missing Flatten layer before Dense layers

Solution

1. Step 1: Analyze model layers

   Flatten converts 2D image to 1D, Dense with 64 units and ReLU is hidden layer, final Dense with 3 units and softmax outputs class probabilities.

2. Step 2: Check compile settings

   Optimizer 'adam' is good, loss 'sparse_categorical_crossentropy' fits multi-class with integer labels, metrics include accuracy.

3. Final Answer:

   Correct setup for multi-class classification -> Option A

4. Quick Check:

   Softmax + sparse_categorical_crossentropy = B [OK]

Hint: Use softmax and sparse_categorical_crossentropy for multi-class [OK]

Common Mistakes:

• Using sigmoid for multi-class output
• Using MSE loss for classification
• Skipping Flatten for image input