home > domains > scrapy-redis.readthedocs.io
Redis-based components for Scrapy.
Distributed crawling/scraping
You can start multiple spider instances that share a single redis queue. Best suitable for broad multi-domain crawls.
Il Mostro
Il Mostro