PHPCrawl 0.81

PHPCrawl is a PHP framework searching the Web. It can be used in writing search crawlers (spiders) that mine Web pages for various information. PHPCrawl acquires information it was configured to fetch and passes it to more powerful apps for further processing.

Features of PHPCrawl:
- Filters for URL and Content-Type data
- Define ways to handle cookies
- Define ways to handle robots.txt files
- Limit its activity in various ways
- Multi-processing modes

Requirements:
- PHP 5 or Higher
- PHP with OpenSSL support

What's New in This Version:
Fixed bugs:
- Links that are partially urlencoded and partially not get rebuild/encoded correctly now.
- Removed a unnecessary debug var_dump() from PHPCrawlerRobotsTxtParser.class.php
- Server-name-indication in TLS/SSL works correctly now.
- "base-href"-tags in websites get interpreted correctly now again.

License type: GNU General Public License
Date added: 3 years, 11 months 28 days ago | Last updated: 2 years, 2 days ago

More popular Search Engine

Listing Files

PHPCrawl_080
libs
UrlCache
ProcessCommunication
List All Files