MCP server

Available tools

The 22 MCP tools exposed by the PyScrappy server.

The server exposes 22 tools. Every tool returns a ScrapeToolResult with the same shape: data (the items), count, scraper, source_urls, and errors.

ToolDescription
scrape_urlScrape any URL: text, links, images, tables, metadata
scrape_wikipediaFetch a Wikipedia article (full / paragraphs / headers)
scrape_stockYahoo Finance quotes, history, and profiles
scrape_newsRSS/Atom feeds, auto-discovered site feeds, or a single article
search_imagesImage search (returns URLs + metadata)
search_youtubeYouTube video search
search_linkedin_jobsPublic LinkedIn job listings
search_githubGitHub repository search (stars, language, …)
search_hackernewsHacker News story search (points, comments)
search_booksBook search via Open Library (title, author, year)
get_weatherCurrent weather for a place (no key)
get_cryptoCryptocurrency prices and market data (CoinGecko)
convert_currencyExchange rates and currency conversion
define_wordWord definitions and examples
search_amazonAmazon product search
search_neweggNewegg electronics / computer hardware search
search_ikeaIKEA furniture / home search
search_soundcloudSoundCloud track search (uses the browser backend)
lookup_movieMovie/TV info from IMDB by title or id (via OMDb; needs OMDB_API_KEY)
scrape_zomatoRestaurant listings by city
search_ubereatsUber Eats restaurants by city
get_ubereats_menuAn Uber Eats restaurant's full menu (from its store URL)

The result envelope

Every tool returns the same typed structure, which gives agents a stable schema:

{
  "data": [ { "...": "item fields depend on the source" } ],
  "count": 5,
  "scraper": "crypto",
  "source_urls": ["https://api.coingecko.com/..."],
  "errors": []
}
  • data: the scraped items. The item shape depends on the source (a movie, a stock quote, an article), so it's a list of free-form objects.
  • errors: non-fatal problems, each with a url and a message.

JavaScript-rendered pages

scrape_url does a fast static fetch by default: it reads the HTML the server sends and does not run JavaScript. Many modern pages (GitHub's contribution graph, dashboards, infinite-scroll feeds) ship an almost-empty HTML shell and build their content with JS in the browser, so a static scrape comes back with little or no data.

When that happens, scrape_url adds a hint to errors telling you the page looks JS-rendered. You have two ways to handle it:

  1. Render the JavaScript. Pass render_js=true so PyScrappy loads the page in a headless browser first, then scrapes the rendered result:

    { "url": "https://github.com/torvalds", "render_js": true }

    This needs the browser extra on the server (pip install 'pyscrappy[browser]' plus playwright install chromium). If it isn't installed, render_js returns an error.

  2. Scrape the data endpoint directly. JS pages fetch their data from an API or a dedicated URL that is static. For example, GitHub's contribution graph lives at https://github.com/users/<name>/contributions, which scrape_url reads with a plain fetch, no browser needed. This is usually faster and more reliable than rendering.

24 scrapers, 22 tools?

Two scrapers aren't exposed as MCP tools because they're blocked without a proxy (Instagram, Twitter/X, Spotify), and the Uber Eats scraper provides two tools (search_ubereats + get_ubereats_menu). The net is 22. See the scraper catalog for the full mapping.