Unit 5: Neural Networks and Deep Learning

Perceptrons, backpropagation and deep architectures

Learning Outcomes
  • Describe the perceptron and multilayer networks
  • Explain forward and backward propagation
  • Identify CNN and RNN applications
  • Outline training and regularisation

Perceptron and Multilayer Networks

A perceptron computes a weighted sum of inputs passed through an activation function. Stacking several layers of neurons forms a multilayer perceptron able to approximate complex non-linear functions.

Training with Backpropagation

A network learns by measuring a loss, propagating its gradient backward through the layers using the chain rule, and updating the weights with gradient descent over many epochs.

from tensorflow.keras import Sequential from tensorflow.keras.layers import Dense model = Sequential([Dense(16, activation="relu"), Dense(1, activation="sigmoid")])

Deep Architectures

Convolutional neural networks use convolution and pooling to excel at images, while recurrent networks and transformers process sequences such as text and time series. Dropout and regularisation curb overfitting.

Summary

This unit introduced neural networks from the perceptron to multilayer and deep architectures, the backpropagation training procedure, and the convolutional and recurrent models used in modern deep learning.

Exercises

  • Explain the role of an activation function in a neuron.
  • Describe the steps of one backpropagation update.
  • State a suitable architecture for image classification and why.
  • Explain how dropout helps reduce overfitting.
← PreviousCourse HomeNext →