---
canonical: "https://www.softatlas.io/scrapfly/products/crawler-api"
name: "Crawler API"
provider: "Scrapfly"
category: "Data & Analytics / Data Integration / Web Scrapers"
review_count: 0
format: "CORE_PRODUCT"
ai_powered: false
website: "https://scrapfly.io/products/crawler-api"
updated: "2026-11-06"
---

# Crawler API

By [Scrapfly](/scrapfly.md)

API that crawls whole websites and returns the collected pages and data.

## Features
- Crawl entire websites
- Follow links with configurable rules
- Respect depth and page limits
- Collect page content in bulk
- Deliver results through webhooks

## Built for
DEVELOPERS

Source: https://www.softatlas.io/scrapfly/products/crawler-api