Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome
GitHub projects from awesome lists
Search awesome repositories
Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.
- Repos indexed
- 21,464
- Awesome lists tracked
- 149
- Current results
- 8
8 repos shown
D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 2 list mentions
- First commit
- 2024-10-13
- History
- 63 history points
- License
- BSD-3-Clause
- Issues
- 9 open
87,954
stars
- Forks
- 9,043
- Commits
- 1,636 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
GitHub topics
- Updated
- 2026-09-10
- Lists
- 2 list mentions
- First commit
- 2009-05-04
- History
- 46 history points
- License
- BSD-3-Clause
- Issues
- 393 open
64,304
stars
- Forks
- 11,948
- Commits
- 11,461 commits
- Star growth, last 7 days
- No 7-day history
- Commit velocity, last 7 days
- No 7-day history
apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
AI dev
GitHub topics
- Updated
- 2026-10-16
- Lists
- 1 list mention
- First commit
- 2016-08-26
- History
- 2 history points
- License
- Apache-2.0
- Issues
- 152 open
26,174
stars
- Forks
- 1,710
- Commits
- 5,789 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
AI dev
GitHub topics
- Updated
- 2026-09-10
- Lists
- 2 list mentions
- First commit
- 2024-01-10
- History
- 43 history points
- License
- Apache-2.0
- Issues
- 96 open
9,508
stars
- Forks
- 805
- Commits
- 1,836 commits
- Star growth, last 7 days
- No 7-day history
- Commit velocity, last 7 days
- No 7-day history
exa-labs/exa-mcp-server
Exa MCP for web search and web crawling!
Stack
GitHub topics
- Updated
- 2026-08-21
- Lists
- 1 list mention
- First commit
- 2024-11-27
- History
- 9 history points
- License
- MIT
- Issues
- 46 open
5,013
stars
- Forks
- 391
- Commits
- 433 commits
- Star growth, last 7 days
- No 7-day history
- Commit velocity, last 7 days
- No 7-day history
webrecorder/browsertrix-crawler
Run a high-fidelity browser-based web archiving crawler in a single Docker container
Stack
GitHub topics
- Updated
- 2026-10-15
- Lists
- 1 list mention
- First commit
- 2020-10-31
- History
- 8 history points
- License
- AGPL-3.0
- Issues
- 146 open
1,166
stars
- Forks
- 152
- Commits
- 769 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
Florents-Tselai/WarcDB
WarcDB: Web crawl data as SQLite databases.
- Updated
- 2024-07-13
- Lists
- 1 list mention
- First commit
- 2022-05-29
- History
- 8 history points
- License
- Apache-2.0
- Issues
- 9 open
407
stars
- Forks
- 10
- Commits
- 73 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
N0taN3rd/Squidwarc
Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
Stack
GitHub topics
- Updated
- 2020-05-19
- Lists
- 1 list mention
- First commit
- 2017-07-20
- History
- 8 history points
- License
- Apache-2.0
- Issues
- 11 open
178
stars
- Forks
- 25
- Commits
- 119 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%