All Perception Tools

Processes XML sitemaps and extracts URLs. Includes features such as support...

Robust web spider for NodeJS

Web Data Scraper - no-code internet scraping. Extract and export to CSV,...

Terminal version of Cambridge Dictionary by default. Also supports the...

arxiv_miner is a toolkit for mining research papers on CS ArXiv.

Ignareo the Carillon, a web crawler/spider template of ultimate high...

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

A web scraping tool to systematically extract the text of scientific papers...

The simple, easy to use command line web crawler.

Official Python SDK for the Supadata API.

Automatically fetch and update proxy lists from multiple sources every 6...

Run Selenium with Python via Github Actions using Headless or Non-Headless browsers!

Discover and verify professional emails using names + domains

🕷️ An easy-to-use spider written in Golang. (previous named GOPA.)

TikTok User Info Scraper allows you to fetch detailed information about...

Updated python-beginners docs and examples

A python library to convert currency using Google Finance.

A library for fetching a list of bot-safe user agents.

Python scripts for building 'Short Jokes' dataset, featured on Kaggle

Twitter Intelligence OSINT project performs tracking and analysis of the Twitter

Set up free and scalable Scrapyd cluster for distributed web-crawling with...

:zap: Ayakashi.io - The next generation web scraping framework

A Python tool for crawling and processing TradingView's PineScript V6...

A simple distributed crawler for zhihu && data analysis

🍇 Club de Programación Creativa: Si algo es automatizable, lo automatizamos;...

Anti-detect browser for web scraping and automation. Engine-level...

A simple client for searching Amazon

The fast, most optimal, and correct HTML & XML parsing library for Python...

🌱 goClone - clone websites in seconds

Puppeteer alternative for Chrome extensions.

Distributed task redisqueue(最简单python分布式函数调度框架)

reborn of https://bitbucket.org/rflechner/scrapysharp

🎠 Run headless Chrome (aka Puppeteer) as a service.

A no-code Google Maps lead scraper built with Google AI Studio (Gemini) and...

Python-based web crawling script with randomized intervals, user-agent...

SDK to access ZenRows API directly from Python. We handle proxies rotation,...

DPULSE - Tool for complex approach to domain OSINT

DART 다트 공시 서류 입수 가공

Web scraper with a simple REST API living in Docker and using a Headless...

Track changes on websites via git

Simple job postings scraper for Indeed based on requests and BeautifulSoup

A python wrapper for the public & private Spotify API

Test runner for Ferret

Social media scraping / data collection tool for the Facebook, Twitter,...

REST API to automate Google NotebookLM - upload sources (URLs, YouTube,...

Web scraper base Pythonic API for Ekşi Sözlük

Powerful and flexible web scraping Search Engine

Crawl all unique internal links found on a given website, and extract SEO...

Experience for effectively fetching Facebook data by Querying Graph API with...

HTTP discovery and change monitoring tool

Free Trial Amazon Scraper API for extracting search, product, offer listing,...

:spider: Google client for SERPS

Image downloader for various imageboards and image albums written in Python.

SDK for Scrapfly.io web scraping API

Grawler is a tool written in PHP which comes with a web interface that...

Monitor instagram user account and automatically post new images to discord...

Automated Aviator Betting Bot for Betika, Spribe & Other Aviator-style sites...

Get all comments from tiktok video url or id

API to get user details for competitive coding platforms - Codeforces,...

Crawly, a high-level web crawling & scraping framework for Elixir.

8 ball pool free rewards collector

Android app for saving webpages for offline reading.

Awdible - Just the best free version of audible. Awdible is a free and...

Command-line utility to convert HTML tables into CSV files

Github follower and following

Scraper for Trading Economics

Lightweight JavaScript library to interact with Chromium-based browsers via...

Intelligent web scraping Claude Code skill with automatic strategy selection...

Biblioteca feita em Python com o objetivo de facilitar o acesso a dados de...

A powerful web scraping library built with Playwright that provides a...

Here is an example of solving reCAPTCHA using the Grid method. In this...

qcrawl - fast async web crawling & scraping framework for Python.

Code relating to scraping public police data.

A CLI toolset to generate table of contents for PDF files automatically.

Download websites and save or view them as markdown, great for feeding into an LLM

An example using Selenium webdrivers for python and Scrapy framework to...

Puppeteer as a service

A libre front-end for Quora

Easy way to brute-force web directory.

Extract list of results from search engines pages as CSV with a bookmarklet...

Download all of a given user's public Pastebin pastes

a golang library and webserver for fast TCP & TLS fingerprinting, powered by eBPF

Scraping of the jobs available and adding them all in one place at peviitor.ro

This repo is used to demo various testing scenarios with Playwright 🎭, using...

A multi-threaded Pakistan Weather crawler written in JavaScript

legacy backend for Open States

Scrapy Training companion code

💵 💰 :brazil: Informações sobre taxas oficiais diárias de Inflação, Selic,...

Google News Scraper, a Python notebook designed to extract news articles...

Facebook Group Members Extractor. Download Facebook group members in CSV.

Scraping Ebay's products using Scrapy Web Crawling Framework

PyTweetToolkit: An intuitive Python library for managing Twitter...

Claude Code Skill for browser automation with Playwright. Model-invoked -...

Web scraper for scraping, tracking and visualizing prices of products on...

Scrapes Spotify and dumps data to a sqlite3 database without auth

Scrapy extension that gives you all the scraping monitoring, alerting,...

Scraping and listing text and image searches on Google, Bing, DuckDuckGo,...

🎯 哔哩哔哩(bilibili)评论区数据可视化分析软件-- up主可用于指导自己的题材选择，明确自己的粉丝群体

Automates the process of downloading all PESU Academy Slides + helps merge...

Structured data gathering from any website using AI-powered scraper,...