All Perception Tools

a python application to track prices on some online shopping applications

Screen scraping and web crawling framework

Crawlyx is an open-source command-line interface (CLI) based web crawler...

Examples and guides for using the Steel API

Your AI friend right in your browser

Proxify is an automated tool that collects and updates fresh SOCKS4, SOCKS5,...

Bulk Downloader for Reddit

Web Scaper for RSI

Get live, population, geography, projected, and historical data from around...

Scrapfly Python SDK for headless browsers and proxy rotation

Spiderbuf 是一个专注于 Python...

A robust and flexible web scraping framework designed to run on Android...

Python library to scrape .docx files with "Entscheidungsbaumdiagramm" tables...

🎼 The clean and modern way of accessing IMSLP data and scores programmatically. 🎶

Golang Crawling and scraping framework

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

简单易用的Python爬虫框架，QQ交流群：597510560

A tool to bypass 2 factor authentication.

Learn Python for the next 30 (or so) Days.

Collection of scripts corresponding to LucidProgramming YouTube tutorials

Spider ported to Python

CLI tool for saving a faithful copy of a complete web page in a single HTML...

An Unofficial API for 1337x, Piratebay, Nyaasi, Torlock, Torrent Galaxy,...

A Python module to scrape several search engines (like Google, Yandex, Bing,...

A base library for building web scrapers for statistical data, and a helper...

aiohttp-like interface to chromium. based on selenium_driverless to bypass cloudflare

Detailed web scraping tutorials for dummies with financial data crawlers on...

Cross Platform C# web crawler framework built for speed and flexibility....

The library provides convenient access to the Outscraper API from...

COVID-19 Coronavirus data scraped from government and curated data sources.

A powerful open-source proxy scraper

🍙 An unofficial scraping tool for https://nautiljon.com, a french anime and...

Generate Free Edu Mail(s) within minutes

This is a Twitter Scraper which uses Selenium for scraping tweets. It is...

Real-time detection of anti-bot systems, CAPTCHAs & fingerprinting...

An undetectable browser automation framework 🤖

Picking stocks through various screening methods. Focus on Northern Europe.

PhantomCrawler is a Python-based web testing and research tool that...

List of major web + mobile browser user agent strings. +1 Bonus script to scrape :)

`scrape_linkedin` is a python package that allows you to scrape personal...

A framework for creating semi-automatic web content extractors

Netflix like full-stack application with SPA client and backend implemented...

Simplify your ETL processes with these hands-on data sanitation tips,...

Example end to end data engineering project.

Kemono Downloader is a fast, powerful PyQt5 app for archiving content from a...

fast cmd-line app that quickly requests millions of urls and can save/echo...

Ingest, parse, and optimize any data format ➡️ from documents to multimedia...

:rocket: Stealth - Secure, Peer-to-Peer, Private and Automateable Web...

A free & open source IMDb front-end.

Serverless IMDB API powered by Cloudflare Worker

Simple but useful Python web scraping tutorial code.

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

Crawl sites for RSS, Atom, and JSON feeds.

The unix-way web crawler

A Python based web scraping api built with fastapi to get manga contents.

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON.

A Chrome extension for writing custom web scraping programs and web...

Attack Surface Discovery tool built on a microservice approach, utilizing...

A fully dynamic and extensible e-commerce scraping framework with automatic...

Pick the most common user-agents on the Internet 👻

This Scrapy project uses Redis and Kafka to create a distributed on demand...

A python program that crawls a website and tries to stress it, polluting...

A python package to generate random request fields for a http header.

Check local static links and online links fast and in parallel

:newspaper: Read RSS feed from LeMonde.fr and display news inside the App

This is a client module to automate the Mozilla Firefox browser via the...

Library designed to replace the SQLite backend by a MongoDB backend on...

A simple Node.js code to get unlimited instagram public pictures by every...

Puppeteer based crawler to measure email and password exfiltration

Multiple chrome extensions that speed up the tasks within MMU CliC.

📰 Build RSS 2.0 feeds from websites (and JSON APIs) automatically or with a...

Learn everything web scraping with David Teather Codes on YouTube

Detect bots, vision AI agents, and headless browsers through 40+ behavioral...

A Python framework to build polite, but tenacious crawlers / scrapers with a...

Python package to generate random proxy on the fly!

Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.

:soccer: Free API with results from national soccer competitions

Home Assistant custom component for scraping (html, xml or json) multiple...

A Python module to scrape data from basketbal-reference.com and convert it...

This whitepaper describes a new concept for building serverless microapps...

Containerized Ferret worker

Official Python library for interacting with Kameleo Client

看理想 Python 客户端 / 下载器，下载看理想的音频和文稿

👒 One Piece TCG data scraper written in Rust

A script that lets you download mangas from mangafox.com

A headless pure-python browser for the web

Локальный API-прокси для Qwen AI с поддержкой сохранения контекста диалогов...

HTTP(S)/SOCKS5 rotating residential proxies - code examples & general information.

Generate Uber Eats accounts using IMAP or Hotmail emails using mobile packets

[Unmaintained] A simple and clean video/music/image downloader 👾

An infinite Pinterest crawler/scraper. Crawl image with inifnite-scroll!

Scraping the TikTok discovery web API every 15 minutes using Github Actions...

Scape top GitHub repositories and users based on keywords

An unofficial API for Quora.

一个灵活、友好的爬虫框架

Chromium via noVNC (Browser in Browser)

AI-powered web scraping CLI. Describe what you want, get a ...

Scrappey.js: A versatile JavaScript wrapper for Scrappey API for solving...

Python framework to scrape Pastebin pastes and analyze them

Learn how to leverage Python's amazing tools to scrape data from other...