Projects Tagged Scrapy: Web Scraping and Crawling Projects Using the Scrapy Framework
Explore projects tagged scrapy — a curated list of projects that use the Scrapy framework for web scraping, data extraction, and large-scale crawling. Discover open-source Scrapy projects and GitHub repositories that demonstrate production-ready spiders, item pipelines, middleware, and distributed crawling patterns (scrapy-redis, scrapy-playwright, Scrapy clusters), along with practical guidance for proxies, request throttling, data pipelines, and deployment on Docker, Kubernetes, or serverless platforms. Use the filtering UI to narrow results by language, license, ecosystem, repository activity, or deployment pattern, then click through to view project details, source code, and contribution guides; filter, compare, and adopt proven Scrapy solutions to accelerate your data collection workflows.
