Introduction

PyScrappy is a Python toolkit for web scraping that works out of the box. Point it at any URL and get structured data back, or reach for one of its built-in scrapers for Wikipedia, Yahoo Finance, news feeds, e-commerce, and more. It also ships an optional MCP server, so AI agents like Claude can call its scrapers as tools.

from pyscrappy import scrape
 
result = scrape("https://en.wikipedia.org/wiki/Web_scraping")
print(result.data[0]["metadata"]["title"])

Highlights

• Generic scraper: give it any URL, get back structured text, links, images, tables, and metadata.
• Auto-pagination: follows "next page" links automatically.
• JS rendering: optional Playwright backend for JavaScript-heavy sites.
• Custom selectors: pass CSS selectors to extract exactly what you need.
• 24 built-in scrapers: Wikipedia, IMDB, stocks, news, GitHub, Hacker News, books, weather, crypto, currency, dictionary, Amazon, Newegg, IKEA, LinkedIn, YouTube, SoundCloud, Zomato, Uber Eats, and more.
• MCP server: expose 22 scrapers as tools for AI agents.
• Concurrent scraping: run many scrapes in parallel with scrape_many and scrape_all.
• Proxy & scraping-API support: route through a proxy or ScraperAPI / ScrapeOps / ScrapingBee for blocked sites.
• Clean, typed API: every scraper returns a ScrapeResult with .to_dataframe() and .to_json().

A quick tour

from pyscrappy import WikipediaScraper, StockScraper, scrape_many, AmazonScraper
 
# A built-in scraper
with WikipediaScraper() as ws:
    article = ws.scrape(query="Python (programming language)", mode="summary")
 
# Finance data as a DataFrame
with StockScraper() as ss:
    df = ss.scrape(symbol="AAPL", mode="history", period="1mo").to_dataframe()
 
# Many queries at once, concurrently
results = scrape_many(AmazonScraper, [{"query": "laptop"}, {"query": "phone"}])

Next steps

• Installation: install PyScrappy and its optional extras.
• Quickstart: your first scrapes, step by step.
• MCP server: use PyScrappy from an AI agent.