Perceptron built from scratch with NumPy and custom neural network trained on Japanese MNIST (70k Hiragana characters) using PyTorch, achieving 91.60% test accuracy through systematic hyperparameter tuning and L2 regularisation.
Built for the Deep Learning subject at UTS, this project had two parts. Part A required implementing a Perceptron from scratch using only NumPy, including forward and backpropagation, without any ML libraries. Part B involved training a fully connected neural network on the Kuzushiji-MNIST dataset, 70,000 handwritten Hiragana characters across 10 classes, using PyTorch.
Four experiments were run progressively: a baseline architecture (89.49% test accuracy, 10.49% overfitting gap), dropout regularisation, systematic hyperparameter tuning (512 neurons, 0.3 dropout, LR 0.001), and a final model extension with L2 regularisation (Weight Decay). The final architecture achieved 91.60% test accuracy with an overfitting gap reduced to 7.14%, selected over the marginally higher-accuracy Experiment 2 for its superior generalisation. A class-level analysis using confusion matrix and F1-scores revealed that Class 2 was the hardest to classify (F1: 0.87), a structural limitation of flattening 2D images for fully connected layers, with CNNs recommended as the natural next step.
