One example to learn all the core concepts of Pytorch. This repo will also work as a training template for any experiment.

In this repo, I tried to implement training routines and inference routines. Along with this, I tried to add various reference links for some of the concepts. Following concepts are implemented in the repo.

• Custom weight initialization
• SAME and VALID padding for Conv2D and Pooling layer in Pytorch just like Tensorflow
• Custom Learning Rate Schedules
• Cosine Anneling Learning Rate
• Learning rate plotting in Tensorboard before starting training.
• Custom implementation of regularization loss
• Model Summary just like Keras
• Reproducibility of each experiments with setting seeds for pytorch and other modules
• TensorBoard Summary Support
• Automatic Mixed Precision Training
• Inference scripts for Pytorch and ONNX
• Custom Dataloader mechanism for dataset handling
• Using pretrained models such as e.g. resnet18
• Restore weights to architecture just like Tensorflow 1.x for pretrained models for finetuning purpose
• Save Checkpoint during training and resume training from that checkpoint
• Remove old checkpoint just like Tensorflow 1.x checkpoint saver mechanism.
• Experiment management: Saving of experiment files in a separate folder during each run.