scrapy ☆

« Back to VersTracker

Description:
Web crawling & scraping framework

Type: Formula | Latest Version: 2.13.4@1 | Tracked Since: Dec 18, 2025

Links: Homepage | @ScrapyProject | formulae.brew.sh

Category: Developer tools

Tags: web-scraping crawling data-extraction python

Install: brew install scrapy

About:
Scrapy is an open-source, collaborative web crawling framework for extracting structured data from websites. It provides a complete toolkit for requests, parsing, and data processing pipelines. Its primary value is enabling rapid development of scalable and maintainable data extraction projects.

Key Features:

Built-in support for asynchronous requests
Interactive shell for rapid prototyping
Extensible architecture with middleware and pipelines
Auto-throttling and politeness controls

Use Cases:

Large-scale data extraction for market analysis
Monitoring website content for changes
Automated data aggregation from multiple sources

Alternatives:

Beautiful Soup – Parsing library focused on HTML/XML parsing, not a full crawling framework
Playwright – Browser automation tool better suited for JavaScript-heavy sites

Version History

Detected	Version	Rev	Change	Commit
Dec 18, 2025 8:22am	2.13.4	1	VERSION_BUMP	a543dd75
Oct 27, 2025 9:43pm		0	VERSION_BUMP	3ed4051a
Oct 19, 2025 10:07pm		0	VERSION_BUMP	0469547b
Oct 19, 2025 1:39pm		0	VERSION_BUMP	3b9e4ff6
Sep 15, 2025 10:04pm		0	VERSION_BUMP	4b38d341
Nov 18, 2024 9:57am		0	VERSION_BUMP	644d7144
Sep 14, 2024 5:12pm		3	VERSION_BUMP	13077fb3