Repositories list

• shub-workflow

  Public
  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •15•13•2•2•Updated Feb 26, 2025Feb 26, 2025

• python-scrapinghub

  Public
  A client interface for Scrapinghub's API

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •63•205•25•4•Updated Feb 21, 2025Feb 21, 2025

• price-parser

  Public
  Extract price amount and currency symbol from a raw text string

  hacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •51•322•16•9•Updated Feb 13, 2025Feb 13, 2025

• scrapinghub-entrypoint-scrapy

  Public
  Scrapy entrypoint for Scrapinghub job runner

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •16•25•8•1•Updated Feb 12, 2025Feb 12, 2025

• scrapy-poet

  Public
  Page Object pattern for Scrapy

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •28•120•12•4•Updated Feb 12, 2025Feb 12, 2025

• scrapinghub-stack-scrapy

  Public
  Software stack with latest Scrapy and updated deps

  Dockerfile

  •

  BSD 3-Clause "New" or "Revised" License

  •20•63•2•2•Updated Feb 11, 2025Feb 11, 2025

• web-poet

  Public
  Web scraping Page Objects core library

  pythonweb-scrapinghacktoberfest+ 1page-objects

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •15•96•18•13•Updated Feb 10, 2025Feb 10, 2025

• andi

  Public
  Library for annotation-based dependency injection

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •5•22•4•1•Updated Feb 7, 2025Feb 7, 2025

• frontera

  Public
  A scalable frontier for web crawlers

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •218•1.3k•79•19•Updated Feb 5, 2025Feb 5, 2025

• extruct

  Public
  Extract embedded metadata from HTML markup

  microformatssemantic-webrdfa+ 4json-ldmicrodataopengraphhacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •116•893•38•14•Updated Feb 5, 2025Feb 5, 2025

• hcf-backend

  Public
  Crawl Frontier HCF backend

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •5•7•2•2•Updated Feb 5, 2025Feb 5, 2025

• dateparser

  Public
  python parser for human readable dates

  hacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •470•2.6k•295•50•Updated Feb 5, 2025Feb 5, 2025

• spidermon

  Public
  Scrapy Extension for monitoring spiders execution.

  testingmonitoringscraping+ 5crawlingspidershacktoberfestmonitoring-toolscrapinghub

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •100•539•42•7•Updated Dec 10, 2024Dec 10, 2024

• scrapy-frontera

  Public
  More flexible and featured Frontera scheduler for Scrapy

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •5•36•2•2•Updated Nov 29, 2024Nov 29, 2024

• social-app-django

  Public
  Python Social Auth - Application - Django

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •385•2•0•1•Updated Nov 18, 2024Nov 18, 2024

• Formasaurus

  Public
  Formasaurus tells you the type of an HTML form and its fields using machine learning

  HTML

  •48•7•1•0•Updated Nov 7, 2024Nov 7, 2024

• number-parser

  Public
  Parse numbers written in natural language

  hacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •25•109•13•6•Updated Oct 23, 2024Oct 23, 2024

• python-crfsuite

  Public
  A python binding for crfsuite

  data-sciencecrfcrfsuite

  Python

  •

  MIT License

  •222•772•45•3•Updated Oct 1, 2024Oct 1, 2024

• streamparse

  Public
  streamparse lets you run Python code against real-time streams of data. Integrates with Apache Storm.

  Python

  •

  Apache License 2.0

  •218•2•0•1•Updated Sep 20, 2024Sep 20, 2024

• splash

  Public
  Lightweight, scriptable browser as a service with an HTTP API

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •514•4.1k•374•26•Updated Aug 2, 2024Aug 2, 2024

• pgcontents

  Public
  A Postgres-backed ContentsManager implementation for IPython

  Python

  •

  Apache License 2.0

  •85•2•0•1•Updated Jul 18, 2024Jul 18, 2024

• shublang

  Public
  Pluggable DSL that uses pipes to perform a series of linear transformations to extract data

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •8•15•23•6•Updated Jul 9, 2024Jul 9, 2024

• woodpecker

  Public
  An opinionated fork of the Drone CI system

  Go

  •

  Other

  •395•0•0•5•Updated Jul 7, 2024Jul 7, 2024

• varanus

  Public
  A command line spider monitoring tool

  spidermonitoringpython36

  Python

  •7•8•2•2•Updated Jul 6, 2024Jul 6, 2024

• scrapyrt

  Public
  HTTP API for Scrapy spiders

  pythoncrawlerscraper+ 7crawlingtwistedscrapywebcrawlerhacktoberfestwebcrawlinghacktoberfest2021

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •161•850•24•6•Updated Jun 28, 2024Jun 28, 2024

• portia

  Public
  Visual scraping for Scrapy

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •1.4k•9.4k•111•19•Updated Jun 26, 2024Jun 26, 2024

• sklearn-crfsuite

  Public
  scikit-learn inspired API for CRFsuite

  Python

  •215•2•0•0•Updated Jun 18, 2024Jun 18, 2024

• webstruct-demo

  Public
  HTTP demo for https://github.com/scrapinghub/webstruct

  Python

  •

  MIT License

  •2•4•0•3•Updated Jun 17, 2024Jun 17, 2024

• autologin

  Public
  A project to attempt to automatically login to a website given a single seed

  Python

  •

  Apache License 2.0

  •43•11•0•2•Updated Jun 17, 2024Jun 17, 2024

• python-intercom

  Public
  Python wrapper for the Intercom API.

  Python

  •

  Other

  •146•1•0•1•Updated Jun 17, 2024Jun 17, 2024