As deployer/Maintainer I should be able to improve Handwritten Digits model Accuracy by experimenting with open and synthetic datasets - Iteration 3

[dileep-gadiraju]

As deployer/Maintainer I should be able to improve Handwritten Digits Accuracy by experimenting with open and synthetic datasets - Iteration 3. Expected accuracy improvement is approximately 3% to 5%.

• Experiment with the existing model version (https://github.com/Sunbird-Saral/Project-Saral/blob/main/v1.0/frontend/SaralApp/android/saralsdk/src/main/assets/trained_resnet_real_synthetic_v2_20.tflite) to identify the miss-classifications.
• Identifying open data sets
• Experiment with new data with the existing model and check the results and Prepare the new data
• Training the next version of the model
• Inferencing and identifying optimal model version/checkpoint.
• Package with SaralSDK/SaralReference App to test in real-time.

[venky3692]

@dileep-gadiraju (Dec 13) - Changed and tested the code for save model checkpoint (only save weights parameter). Trained the model for 5 epochs with existing digits dataset with data augmentation. Inference done on the same test dataset.
Accuracy on NSIT dataset (~60%). Almost same to old checkpoint.
Accuracy on Not in MNIST dataset (~97%). 3% increase than old checkpoint.

[venky3692]

@dileep-gadiraju (Dec 14)

• Tested new model on the existing dataset. Gives same results. Max 0.3% - 05% difference in accuracy for some classes
• Analysed the miss-classifications on the NSIT dataset with images drawn in the notebook. Seems like the model just fails to understand the variations between different digits.
• To segregate training images (NSIT dataset) for the model, tried pixel counting of (0,0,0) pixels to eliminate darkened numbers, unwanted lines around numbers.