Skip to content

Navigation Menu

Appearance settings

scrapy

Explore
By company size
By use case
By industry
View all solutions
Topics
- AI
- DevOps
- Security
- Software Development
- View all
Explore
- GitHub Sponsors
  Fund open source developers
- The ReadME Project
  GitHub community articles
Repositories
- Enterprise platform
  AI-powered developer platform
Available add-ons
Pricing

Search code, repositories, users, issues, pull requests...

Search

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

Scrapy project

An open source and collaborative framework for extracting the data you need from websites. In a fast, simple, yet extensible way.

614 followers
https://scrapy.org

Overview
Repositories
Projects
Packages
People

More

Overview
Repositories
Projects
Packages
People

Pinned Loading

scrapy scrapy Public

Scrapy, a fast high-level web crawling & scraping framework for Python.

Python 55.3k 10.8k
scrapy.org scrapy.org Public

The scrapy.org website

HTML 64 142

Repositories

Loading

Type

Select type

All Public Sources Forks Archived Mirrors Templates

Language

Select language

All C++ DIGITAL Command Language HTML Python Shell

Sort

Select order

Last updated Name Stars

Showing 10 of 27 repositories

scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.

scrapy/scrapy’s past year of commit activity

Python 55,323 BSD-3-Clause 10,815 449 (19 issues need help) 197 Updated May 23, 2025
scrapyd-client Public
Command line client for Scrapyd server

scrapy/scrapyd-client’s past year of commit activity

Python 772 BSD-3-Clause 146 5 0 Updated May 22, 2025
parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

scrapy/parsel’s past year of commit activity

Python 1,229 BSD-3-Clause 151 32 (1 issue needs help) 12 Updated May 12, 2025
scrapy.org Public
The scrapy.org website

scrapy/scrapy.org’s past year of commit activity

HTML 64 142 1 1 Updated May 8, 2025
cssselect Public
CSS Selectors for Python

scrapy/cssselect’s past year of commit activity

Python 297 60 18 5 Updated May 5, 2025
w3lib Public
Python library of web-related functions

scrapy/w3lib’s past year of commit activity

Python 405 BSD-3-Clause 107 11 (1 issue needs help) 4 Updated May 5, 2025
scrapyd Public
A service daemon to run Scrapy spiders

scrapy/scrapyd’s past year of commit activity

Python 3,031 BSD-3-Clause 571 9 0 Updated Apr 12, 2025
queuelib Public
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

scrapy/queuelib’s past year of commit activity

Python 277 BSD-3-Clause 55 3 2 Updated Mar 31, 2025
protego Public
A pure-Python robots.txt parser with support for modern conventions.

scrapy/protego’s past year of commit activity

DIGITAL Command Language 65 BSD-3-Clause 28 5 (1 issue needs help) 0 Updated Mar 24, 2025
itemloaders Public
Library to populate items using XPath and CSS with a convenient API

scrapy/itemloaders’s past year of commit activity

Python 48 BSD-3-Clause 15 17 5 Updated Mar 24, 2025

View all repositories

People

Top languages

Loading…

Uh oh!

There was an error while loading. Please reload this page.

Most used topics

Loading…

Uh oh!

There was an error while loading. Please reload this page.

Footer

© 2025 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Docs
Contact

You can’t perform that action at this time.