Organizations by Tag: crawler — Teams Using Web Crawlers for Data Extraction, Indexing, and Distributed Crawling
Explore organizations tagged with crawler that design and operate web crawlers and data-extraction platforms, from Scrapy and Puppeteer-based scrapers to distributed indexing systems and real-time ingestion pipelines. This curated list (organizations > tags) explains how these teams leverage headless browsers, proxy rotation, rate-limiting, sitemap parsing, incremental crawling, deduplication, and cloud-scale orchestration to power search, analytics, and market-intelligence use cases while maintaining compliance and politeness. Use the filtering UI to narrow results by technology, scale, or industry, compare implementation patterns and tooling (Scrapy, Apache Nutch, Puppeteer, Heritrix), view profiles and code examples, and contact teams to accelerate your crawler-driven data projects.
