Description:
Web crawling & scraping framework
|
|
Type: Formula
|
Latest Version: 2.13.4@1
|
Tracked Since: Dec 18, 2025
|
|
Links:
Homepage |
@ScrapyProject |
formulae.brew.sh
|
|
Category: Developer tools
|
|
Tags:
web-scraping
crawling
data-extraction
python
|
|
Install:
brew install scrapy
|
About:
Scrapy is an open-source, collaborative web crawling framework for extracting structured data from websites. It provides a complete toolkit for requests, parsing, and data processing pipelines. Its primary value is enabling rapid development of scalable and maintainable data extraction projects.
|
Key Features:
- Built-in support for asynchronous requests
- Interactive shell for rapid prototyping
- Extensible architecture with middleware and pipelines
- Auto-throttling and politeness controls
|
Use Cases:
- Large-scale data extraction for market analysis
- Monitoring website content for changes
- Automated data aggregation from multiple sources
|
Alternatives:
-
Beautiful Soup
– Parsing library focused on HTML/XML parsing, not a full crawling framework
-
Playwright
– Browser automation tool better suited for JavaScript-heavy sites
|
| Detected |
Version |
Rev |
Change |
Commit |
| Dec 18, 2025 8:22am |
2.13.4 |
1 |
VERSION_BUMP |
a543dd75 |
| Oct 27, 2025 9:43pm |
|
0 |
VERSION_BUMP |
3ed4051a |
| Oct 19, 2025 10:07pm |
|
0 |
VERSION_BUMP |
0469547b |
| Oct 19, 2025 1:39pm |
|
0 |
VERSION_BUMP |
3b9e4ff6 |
| Sep 15, 2025 10:04pm |
|
0 |
VERSION_BUMP |
4b38d341 |
| Nov 18, 2024 9:57am |
|
0 |
VERSION_BUMP |
644d7144 |
| Sep 14, 2024 5:12pm |
|
3 |
VERSION_BUMP |
13077fb3 |
| May 22, 2024 9:53am |
|
1 |
VERSION_BUMP |
6c47d5d6 |
| May 22, 2024 4:04am |
|
1 |
VERSION_BUMP |
56645c56 |
| May 14, 2024 12:44pm |
|
0 |
VERSION_BUMP |
68bf34dd |
| Feb 15, 2024 7:48am |
|
0 |
VERSION_BUMP |
f15b60ac |
| Oct 6, 2023 5:34pm |
|
1 |
VERSION_BUMP |
3aba5448 |
| Sep 24, 2023 1:37pm |
|
0 |
VERSION_BUMP |
2da97e40 |
| Aug 30, 2023 12:55pm |
|
0 |
VERSION_BUMP |
c8b32bde |
| Aug 30, 2023 7:24am |
|
0 |
VERSION_BUMP |
1c4a47b3 |
| Aug 6, 2023 3:27am |
|
0 |
VERSION_BUMP |
a3af08b3 |
|