Scrapyd

Scrapy projects:

This web UI is for monitoring only. To upload projects and schedule crawls, use the API. For example, using curl:

curl http://localhost:6800/schedule.json -d project=default -d spider=somespider

See the Scrapyd documentation for details.