'scrapy fail to scrap all the items

There are more than 500 items, but scrapy shell only manages 5 items.

scrapy shell

from urllib import response
import scrapy


class Elo1Spider(scrapy.Spider):
name = 'elo1'
allowed_domains = ['exportleftovers.com']
start_urls = ['http://exportleftovers.com/']

def parse(self, response):
    for products in response.css('div.product-wrap'):
        yield {
            'name':products.css('a.product-thumbnail__title::text').get() ,
            'price' : products.css('span.money::text').get().strip(),
              }
        next_page = response.css('a.pagination-next').attrib['href']
        if next_page is not None:

            yield response.follow(next_page,callback=self.parse)

    


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source