Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 21,464
Awesome lists tracked: 149
Current results: 18

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Unsure which signals matter? Read the repository shortlisting guide.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

18 repos shown

Topic: scraper

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

firecrawl/firecrawl

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

AI dev

Stack

TypeScript Astro Express Next.js pytest Bundler Cargo Composer

GitHub topics

#ai #ai-agents #ai-crawler #ai-scraping #ai-search #crawler

Updated: 2026-10-17
Lists: 2 list mentions
First commit: 2024-04-15
History: 183 history points
License: AGPL-3.0
Issues: 619 open

192,247

stars

Forks: 10,155
Commits: 6,571 commits
Star growth, last 7 days: +1,995 +1.0%
Commit velocity, last 7 days: +26 +0.4%

Website GitHub

unclecode/crawl4ai

Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.

AI dev

Stack

Python FastAPI Flask pip uv

GitHub topics

#ai #ai-agents #crawler #data-extraction #llm #markdown

Updated: 2026-10-15
Lists: 3 list mentions
First commit: 2024-05-09
History: 174 history points
License: Apache-2.0
Issues: 261 open

85,680

stars

Forks: 8,905
Commits: 1,778 commits
Star growth, last 7 days: +511 +0.6%
Commit velocity, last 7 days: +68 +4.0%

Website GitHub

huginn/huginn

Create agents that monitor and act on your behalf. Your agents are standing by!

Stack

Ruby Ruby on Rails Svelte Bundler npm

GitHub topics

#agent #automation #feed #feedgenerator #huginn #monitoring

Updated: 2026-09-26
Lists: 3 list mentions
First commit: 2013-03-10
History: 9 history points
License: MIT
Issues: 699 open

50,019

stars

Forks: 4,302
Commits: 4,208 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

feder-cr/invisible_playwright_mcp

Playwright MCP server undetected by anti-bots and captchas: AI agent browses the web on anti-detect stealth Firefox, Python, undetected browser automation, scraping, computer use.

AI dev

Stack

Python pytest Starlette PEP 517

GitHub topics

#agentic-ai #ai-agent #ai-automation #autonomous-agents #browser #claude-code-plugin

Updated: 2026-10-06
Lists: 2 list mentions
First commit: 2024-12-05
History: 34 history points
License: MIT
Issues: 2 open

31,777

stars

Forks: 4,689
Commits: 364 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

iawia002/lux

👾 Fast and simple video download library and CLI tool written in Go

Stack

Go Go modules

GitHub topics

#bilibili #crawler #download #downloader #go #golang

Updated: 2026-03-29
Lists: 1 list mention
First commit: 2018-02-06
History: 5 history points
License: MIT
Issues: 545 open

31,685

stars

Forks: 3,312
Commits: 869 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

cheeriojs/cheerio

The fast, flexible, and elegant library for parsing and manipulating HTML and XML.

Stack

TypeScript Astro React Tailwind CSS Vite npm

GitHub topics

#cheerio #dom #hacktoberfest #html #htmlparser #htmlparser2

Updated: 2026-10-12
Lists: 1 list mention
First commit: 2011-10-05
History: 1 history point
License: MIT
Issues: 65 open

30,515

stars

Forks: 1,728
Commits: 4,450 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

AI dev

Stack

TypeScript Express React npm pnpm Yarn

GitHub topics

#apify #automation #crawler #crawling #headless #headless-chrome

Updated: 2026-10-16
Lists: 1 list mention
First commit: 2016-08-26
History: 2 history points
License: Apache-2.0
Issues: 152 open

26,174

stars

Forks: 1,710
Commits: 5,789 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apify/crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

AI dev

Stack

Python pytest React npm PEP 517 pip

GitHub topics

#apify #automation #beautifulsoup #crawler #crawling #headless

Updated: 2026-09-10
Lists: 2 list mentions
First commit: 2024-01-10
History: 43 history points
License: Apache-2.0
Issues: 96 open

9,508

stars

Forks: 805
Commits: 1,836 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

mishushakov/llm-scraper

Turn any webpage into structured data using LLMs

Stack

TypeScript Vite npm

GitHub topics

#ai #artificial-intelligence #browser #browser-automation #gpt #gpt-4

Updated: 2026-08-02
Lists: 2 list mentions
First commit: 2024-04-20
History: 1 history point
License: MIT
Issues: 7 open

6,915

stars

Forks: 453
Commits: 108 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

d60/twikit

Stack

Python pip

GitHub topics

#bot #client #python #python-web-scraper #python3 #scrape

Updated: 2026-03-10
Lists: 0 list mentions
First commit: 2024-12-01
History: 46 history points
License: MIT
Issues: 160 open

4,733

stars

Forks: 573
Commits: 39 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

myreader-io/myGPTReader

A community-driven way to read and chat with AI bots - powered by chatGPT.

Archived

Stack

Python Flask LangChain LlamaIndex React npm pip

GitHub topics

#ai #chatgpt #crawler #daily-news #embedding #gpt-35-turbo

Updated: 2026-02-05
Lists: 1 list mention
First commit: 2023-03-08
History: 8 history points
License: MIT
Issues: 15 open

4,412

stars

Forks: 435
Commits: 305 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

itsOwen/CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Stack

Python LangChain Streamlit pip

GitHub topics

#ai-scraping #gemini-api #llm #llm-scraper #openai #scraper

Updated: 2026-08-20
Lists: 1 list mention
First commit: 2024-08-17
History: 1 history point
License: MIT
Issues: 3 open

3,252

stars

Forks: 356
Commits: 107 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

felipecsl/wombat

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Stack

Ruby Bundler

GitHub topics

#crawler #dsl #ruby #scraper

Updated: 2026-04-07
Lists: 1 list mention
First commit: 2011-11-14
History: 6 history points
License: MIT
Issues: 24 open

1,359

stars

Forks: 128
Commits: 300 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

0x676e67/wreq

An ergonomic, privacy-aware Rust HTTP Client

Stack

Rust Cargo

GitHub topics

#akamai #censorship-resistant #client #crawler #fingerprint #http

Updated: 2026-10-15
Lists: 1 list mention
First commit: 2016-07-01
History: 8 history points
License: Apache-2.0
Issues: 27 open

1,068

stars

Forks: 140
Commits: 2,943 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

openzim/zimit

Make a ZIM file from any Web site and surf offline!

Stack

Python pytest PEP 517

GitHub topics

#docker #scraper #webscraping #zim

Updated: 2026-07-31
Lists: 1 list mention
First commit: 2016-01-09
History: 7 history points
License: GPL-3.0
Issues: 90 open

844

stars

Forks: 50
Commits: 518 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

postmodern/spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Stack

Ruby Sinatra Bundler

GitHub topics

#crawler #ruby #scraper #spider #spider-links #web

Updated: 2026-01-12
Lists: 1 list mention
First commit: 2008-05-19
History: 6 history points
License: MIT
Issues: 16 open

836

stars

Forks: 105
Commits: 898 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

Cybrarist/Discount-Bandit

A Multi-User Selfhosted price tracker for Amazon, Aliexpress, ebay and many more along with custom stores. Get notified when price matches one or more criterias set by the user

Stack

PHP Laravel Tailwind CSS Vite Composer npm

GitHub topics

#amazon-price-tracker #automation #ebay #ecommerce #filamentphp #laravel

Updated: 2026-06-01
Lists: 1 list mention
First commit: 2023-07-06
History: 8 history points
License: Unknown
Issues: 14 open

745

stars

Forks: 41
Commits: 221 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

geeks-of-data/knowledge-gpt

Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.

Stack

Python FastAPI pip

GitHub topics

#context #embedding #embedding-vectors #gpt #gpt3-turbo #gpt4

Updated: 2023-04-25
Lists: 1 list mention
First commit: 2023-03-11
History: 7 history points
License: MIT
Issues: 8 open

292

stars

Forks: 52
Commits: 163 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing