How to resize(reshape) the images in CNN? Mathematical intuition behind resizing

Question

I have been working on Images for few months for my internship, and recently I have been wondering that is there a mathematical way of resizing the images. This becomes a fairly difficult task to resize the images because many a times freshers like me have little experience about the pre-processing in Images. Given that my problem statement was Gender classification using the human eye. However I found it difficult because

1. The images were 3 channel
2. The images were in rectangular shape (17:11)
3. I did try to resize the images by following few blogs which said to start small and then go up, while it could have worked I still did not understand how small. I resized them to 800,800 randomly and go Resource Exhaustive error(I was using GPU).

So I ask the community if there is any such mathematical formula or a generalized way of doing the resizing task. Thank you in advance.

Answer 1

This partially answers your question. But, normally many people use transfer learning and a pre-designed architecture for computer vision tasks. Since almost all architecture is designed for square input shape, you can get a better results by making the shape of your input image squared. Another solution would be only padding your 17X11 to make it square by 0 values. (you need to test to see which one works best in your case, but the common practice is re-shaping to square.)

It is fine to have 3 channel images, almost all images are designed for 3 channel input ( even for BW images it is suggested to repeat the channel to have 3 channel input for the model) About resizing

About resizing the image, in theory, you need to resize the image to the model you are going to use. For example, LeNet-5 accepts images of Mnist with size 28x28. In theory, larger images result in better model performance, but in your case, the images are super low resolution you can start with 28x28 or 224x224 architectures and later use bigger ones and see if it helps in your case.

About the error it's pretty normal your model size was going to be bigger than your GPU memory so, you see Out of memory error. you can use a smaller model ( and smaller input image size) with your device, or you need to use a device with bigger GPU memory.

Finally, you should consider the size of architecture you are going to reuse to determine the correct resize of the dataset you need. If you are designing your model then best starting point can be something around 28x28 ( basically using Lenet) and later developing based on needs/performance.

the resizing can be as easy as calling a Transform with Pytorch transforms like ( i mean you don't need to manually recreate a copy of the dataset just for resizing)

T.Compose([
    T.RandomResize(224)
])