Selectors | PyScrappy

Navigate HTML directly with a chainable CSS/XPath parser, Scrapy/BeautifulSoup-style.

Most of PyScrappy returns structured dicts. When you'd rather traverse the markup yourself, use Selector — a chainable parser with CSS and XPath selection, BeautifulSoup-style search, text search, and structural "find similar".

from pyscrappy import Selector
 
page = Selector(html)                              # or navigate any HTML string
page.css(".title::text").getall()                  # CSS with ::text / ::attr(name)
page.xpath("//a/@href").getall()                    # XPath (elements, text(), @attr)
page.find_all("h2", class_="title")                 # BeautifulSoup-style search
page.find_by_text("Add to cart", tag="button")      # search by text content

Selecting

css() and xpath() return a SelectorList. Read values off it with .get() (first) / .getall() (all) / .text():

first = page.css(".product")[0]
first.css(".price::text").get()        # chainable — select within a selection
page.css(".title").text()              # ["Alpha", "Beta"]

CSS supports a trailing ::text or ::attr(name) pseudo-element, so .get() / .getall() return those strings instead of elements.
XPath text (.../text()) and attribute (.../@href) expressions return strings too.

Finding

page.find_all("a", class_="nav")                    # by tag + class + attributes
page.find_by_text("Sold out")                        # substring match, any tag
page.find_by_text("Sold out", tag="span", exact=True)  # exact text, specific tag

Find similar

Located one card or row and want the rest? find_similar() returns sibling elements with the same tag and overlapping classes:

first_card = page.css(".product")[0]
all_cards = first_card.find_similar()                # every sibling product tile

Self-healing selectors

Selectors can also survive site markup changes — see Adaptive selectors.