I am doing the image classification problem on medical imaging dataset, but the caveat is that the total no. of samples is very low(<1000). I am using transfer learning(pretrained Dense Net 121) model and have applied some image augmentation techniques on train data(random rotation, flipping etc.) After training for around 60 epochs, using lr of 0.001 , and also applied weight decay of 1e-5 , the model is getting overfitted. (I know, since the number of observations are very less, hence it ought to be the case), but even after doing augmentation and weight decay, the max validation accuracy I am getting is ~77% . Even after increasing the batch size from 16 to 32 , there are no impressible results.
How to solve this ?