A Beginner's Guide to PyTorch's nn.Sequential for Neural Network Architecture Design

We can create the deep neural network, convolutional neural network, and other neural networks using the Pytorch library, torch.nn.

First, let's import the necessary libraries:

import torch
import torch.nn as nn

Example 1: Creating a simple feedforward neural network with two hidden layers and ReLU activation

model = nn.Sequential(
          nn.Linear(784, 256),  # input layer -> hidden layer 1
          nn.ReLU(),  # activation function
          nn.Linear(256, 128),  # hidden layer 1 -> hidden layer 2
          nn.ReLU(),  # activation function
          nn.Linear(128, 10)  # hidden layer 2 -> output layer
        )

In the example above, we are creating a simple feedforward neural network with two hidden layers and a ReLU activation function. The input layer has 784 nodes (corresponding to a 28x28 pixel image), the first hidden layer has 256 nodes, the second hidden layer has 128 nodes, and the output layer has 10 nodes (corresponding to 10 possible classes in a classification task).

Example 2: Creating a convolutional neural network (CNN) with two convolutional layers, max pooling, and dropout

model = nn.Sequential(
          nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),  # input layer -> conv layer 1
          nn.ReLU(),  # activation function
          nn.MaxPool2d(kernel_size=2),  # max pooling layer
          nn.Dropout(p=0.25),  # dropout layer
          nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1),  # conv layer 1 -> conv layer 2
          nn.ReLU(),  # activation function
          nn.MaxPool2d(kernel_size=2),  # max pooling layer
          nn.Dropout(p=0.25),  # dropout layer
          nn.Flatten(),  # flatten the output of the convolutional layers
          nn.Linear(64 * 7 * 7, 512),  # fully connected layer 1
          nn.ReLU(),  # activation function
          nn.Dropout(p=0.5),  # dropout layer
          nn.Linear(512, 10)  # fully connected layer 2 -> output layer
        )

In the example above, we are creating a CNN with two convolutional layers, max pooling, and dropout. The first convolutional layer has 32 filters of size 3x3, the second convolutional layer has 64 filters of size 3x3, and both layers have a padding of 1 to maintain the spatial dimensions. After each convolutional layer, we apply a ReLU activation function, followed by a max pooling layer with kernel size 2x2. We then apply dropout with a probability of 0.25 to regularize the model. Finally, we flatten the output of the convolutional layers and pass it through two fully connected layers with ReLU activation and dropout, before outputting the final classification result.

Example 3: Creating a convolutional neural network (CNN) with three convolutional layers with batch normalization and ReLU activation, followed by two fully connected layers with ReLU activation.

import torch
import torch.nn as nn

class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        
        self.conv1 = nn.Conv2d(1, 16, kernel_size=3, padding=1)
        self.bn1 = nn.BatchNorm2d(16)
        self.relu1 = nn.ReLU(inplace=True)
        
        self.conv2 = nn.Conv2d(16, 32, kernel_size=3, padding=1)
        self.bn2 = nn.BatchNorm2d(32)
        self.relu2 = nn.ReLU(inplace=True)
        
        self.conv3 = nn.Conv2d(32, 64, kernel_size=3, padding=1)
        self.bn3 = nn.BatchNorm2d(64)
        self.relu3 = nn.ReLU(inplace=True)
        
        self.fc1 = nn.Linear(64 * 7 * 7, 128)
        self.relu4 = nn.ReLU(inplace=True)
        
        self.fc2 = nn.Linear(128, 10)
    
    def forward(self, x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu1(x)
        
        x = self.conv2(x)
        x = self.bn2(x)
        x = self.relu2(x)
        
        x = self.conv3(x)
        x = self.bn3(x)
        x = self.relu3(x)
        
        x = x.view(-1, 64 * 7 * 7)
        
        x = self.fc1(x)
        x = self.relu4(x)
        
        x = self.fc2(x)
        
        return x

Example 4: Creating a Deep Neural Network (DNN) architecture consisting of two fully connected layers with ReLU activation.

import torch
import torch.nn as nn

class DNN(nn.Module):
    def __init__(self):
        super(DNN, self).__init__()
        
        self.fc1 = nn.Linear(784, 512)
        self.relu1 = nn.ReLU(inplace=True)
        
        self.fc2 = nn.Linear(512, 256)
        self.relu2 = nn.ReLU(inplace=True)
        
        self.fc3 = nn.Linear(256, 10)
    
    def forward(self, x):
        x = x.view(-1, 784)
        
        x = self.fc1(x)
        x = self.relu1(x)
        
        x = self.fc2(x)
        x = self.relu2(x)
        
        x = self.fc3(x)
        
        return x

Example 5: Creating a Reinforcement Learning Architecture

import torch
import torch.nn as nn

class ReinforcementLearning(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(ReinforcementLearning, self).__init__()
        
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu1 = nn.ReLU(inplace=True)
        
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.relu2 = nn.ReLU(inplace=True)
        
        self.fc3 = nn.Linear(hidden_size, output_size)
    
    def forward(self, x):
        x = self.fc1(x)
        x = self.relu1(x)
        
        x = self.fc2(x)
        x = self.relu2(x)
        
        x = self.fc3(x)