How to load images from URL using Pytorch

Question

I want to load the images using Pytorch

I have a dataset of image_urls with its corresponding labels(offer_id are labels.)

Is there any efficient way of doing it in Pytorch?.

Answer 1

You may convert your image URLs to files first by downloading them to specific folder representing the label. You will certainly find a way to do so. Then you may do like this to check what you have:

%%time
import glob
f=glob.glob('/content/imgs/**/*.png')
print(len(f), f)

There is a need to create a image loader that will read the image from disk. In here the pil_loader:

def pil_loader(path):    
    with open(path, 'rb') as f:
        img = Image.open(f)
        return img.convert('RGB')


ds = torchvision.datasets.DatasetFolder('/content/imgs', 
                                        loader=pil_loader, 
                                        extensions=('.png'), 
                                        transform=t)

print(ds)

You may check how I did that for Cifar10.

Check the section "From PNGs to dataset".

Answer 2

You can use the requests package:

import requests
from PIL import Image
import io
import cv2
response = requests.get(df1.URL[0]).content
im = Image.open(io.BytesIO(response))