EXP-03 · deep learning
Does it read my handwriting?
Architecture search across MLPs and CNNs, a confidence interval for accuracy, a production model, and a test on digits I wrote by hand.
Question
Does a 99% MNIST model still work on digits I drew myself, outside the dataset?
Problem
Classify handwritten digits 0–9 and check that the model generalizes beyond the dataset.
Data
MNIST: 70,000 28×28 grayscale images of digits 0–9 (60,000 train, 10,000 test), loaded through Keras. Plus 10 digits I drew myself to test the model outside the dataset.
Method
EDA, stratified splits, a 32-configuration MLP/CNN grid without Keras Tuner, a 95% Wilson CI, 200-feature selection by mutual information, retraining on train+val, and OpenCV preprocessing for my own images.
Result
99.41% test accuracy (95% CI 99.22–99.52% for the production model), and all 10 digits I wrote by hand recognized correctly.
What I learned
Even with 200 of 784 pixels it keeps 97.4%: most pixels are noise.
Stack
- Python
- TensorFlow
- Keras
- scikit-learn
- OpenCV
Context
Course: Machine Learning III · 2025