Data as of Oct 20, 2026Based on 30,145 AI responses
Reviewed by Dimitry Apollonsky
Scrapyd is a service for deploying and running Scrapy spiders. It enables uploading Scrapy projects (as Python eggs) and controlling their spiders via a JSON API, including scheduling crawls with endpoints such as schedule.json. Projects are deployed by building an egg and uploading it through addversion.json (via scrapyd-deploy), and Scrapyd can be run in Docker with a web/API interface and endpoints for managing projects, versions, spiders, and jobs (e.g., listprojects.json, listspiders.json, listjobs.json).
Hosted on Read the Docs
<1%
across Scrapyd’s markets · since Jul 5
The market map
Web Scraping & Browser Automation APIsMentioned in
Each engine’s share of its own answers across Scrapyd’s markets · since Jul 5
Where Scrapyd ranks in AI