Is there a good rule out there to choose an appropriate batch size?

Question

The common heuristic is to ensure it is as big as your acceleration hardware allows it. Nevertheless, when training on a small dataset with a large batch size it appears that the training is inefficient because each epoch only has a small number of steps, which would prevent a good enough number of updates in order to make the training converge.