Comparative study of CNN transfer learning on 101-class food image classification, with MobileNetV3 achieving 72.44% Top-1 accuracy after deep fine-tuning across GoogLeNet, MobileNetV3, and ResNet50.
Built for the Deep Learning subject at UTS, this project explored transfer learning and fine-tuning strategies on the Food-101 dataset, 101,000 real-world food images across 101 categories. Three pre-trained CNN architectures were evaluated under identical conditions: GoogLeNet (46.93% Top-1), ResNet50 (50.93%), and MobileNetV3 (60.27%). Each model had its original head replaced with a custom three-layer fully connected classifier trained on frozen ImageNet weights. MobileNetV3 was selected for fine-tuning due to its superior baseline accuracy and parameter efficiency (4.5M parameters vs 26M for ResNet50).
Two fine-tuning experiments were run using differential learning rates: unfreezing the final projection block (features[-1]) pushed accuracy to 65.94%, while unfreezing the last three Inverted Residual blocks (features[-4:]) achieved the final Top-1 accuracy of 72.44% and Top-5 of 91.33%. Error analysis revealed that the hardest misclassifications occur between visually similar dishes like steak vs filet mignon and tuna tartare vs beef tartare, a known limitation of visual-only classification on the Food-101 dataset.
