Skip to content

Crawl & Search

Guide: Crawl Web & Search Images

Overview

You can use RevidAPI public endpoints to: - Crawl website content (returns markdown/content and embedded images/videos depending on mode). - Search images by keyword (returns an image list and minimal metadata that you can use for thumbnails/suggestions).

Pricing

5 credits per request

Fixed cost per call, not dependent on the length of the response content.

Endpoints

1. Crawl web pages

  • URL: POST https://revidapi.com/v1/website/crawl
  • Method: POST

2. Search images by keyword

  • URL: POST https://revidapi.com/v1/search/image
  • Method: POST

Requirements

Headers (common)

  • x-api-key: Required

Request Body

1. Crawl web (POST /v1/website/crawl)

Parameter Type Required Description
url string ✅ Full URL to crawl (any website / article / landing page).
mode string ❌ Default article: main content (markdown) + embedded images/videos. web: returns only links (URLs found on the page); markdown/content are empty — use it to list articles from a category page, then call each URL with article.

Example requests (cURL)

curl -s -X POST "https://revidapi.com/v1/website/crawl" \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_X_API_KEY" \
  -d '{"url":"https://example.com","mode":"web"}'

mode=article (get main content + images/videos)

curl -s -X POST "https://revidapi.com/v1/website/crawl" \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_X_API_KEY" \
  -d '{"url":"https://example.com/article","mode":"article"}'

2. Search images by keyword (POST /v1/search/image)

Parameter Type Required Description
keyword string ✅ Image search keyword (English or Vietnamese are both OK).
max_results int ❌ Maximum number of images to return (1-100). Default: 20.

Example request (cURL)

curl -s -X POST "https://revidapi.com/v1/search/image" \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_X_API_KEY" \
  -d '{"keyword":"sunset beach","max_results":20}'

Response

1. Crawl web

The response JSON looks like:

{
  "success": true,
  "url": "https://example.com",
  "title": "Article title",
  "markdown": "...",
  "content": "...",
  "images": [{"url":"https://.../image.jpg","alt":"..."}],
  "videos": [{"url":"https://...","type":"embed"}],
  "links": ["https://...","https://..."]
}

Notes by mode: - mode=web: returns links; does not return images (this documentation guarantees no images in mode web). - mode=article: returns markdown/content + images (images inside the article, before the "Related news" section) and videos (embedded video URL if available); always returns links: [].

2. Search images by keyword

Minimal JSON response:

{
  "success": true,
  "keyword": "sunset beach",
  "images": [
    {
      "url": "https://.../image.jpg",
      "title": "Image title",
      "source": "https://source-site.com/article",
      "thumbnail": "https://.../thumb.jpg"
    }
  ],
  "count": 20
}

More search endpoints

Both take a JSON body, return immediately and cost 5 credits per successful call.

3. Search web pages by keyword (POST /v1/search/url)

Parameter Type Required Description
keyword string ✅ Search term
max_results int ❌ How many results to return
curl -s -X POST "https://revidapi.com/v1/search/url" \
  -H "x-api-key: $REVID_KEY" -H "Content-Type: application/json" \
  -d '{"keyword": "claude code", "max_results": 5}'
{
  "success": true,
  "keyword": "claude code",
  "urls": [
    {"url": "https://code.claude.com/", "title": "Claude Code by Anthropic | AI Coding Agent, Terminal, IDE",
     "snippet": "Anthropic's agentic coding tool for developers. ..."}
  ]
}

Pair it with /v1/website/crawl to read each page: search → take urls[].url → crawl with mode: "article".

4. Search stock video clips by keyword (POST /v1/search/videos)

Parameter Type Required Description
keyword string ✅ What the clip should show (English works best)
max_results int ❌ How many clips to return
curl -s -X POST "https://revidapi.com/v1/search/videos" \
  -H "x-api-key: $REVID_KEY" -H "Content-Type: application/json" \
  -d '{"keyword": "city traffic at night", "max_results": 3}'
{
  "success": true,
  "keyword": "city traffic at night",
  "count": 3,
  "videos": [
    {"url": "https://videos.pexels.com/video-files/.../...mp4", "duration_seconds": 3.0}
  ]
}

Clips come from free stock libraries; use them as B-roll in Video Editing. Check each library's licence before commercial use.

Error Responses

Status Description
400 Invalid request (missing/wrong parameter format).
401 Invalid x-api-key.
429 Overloaded / queue limit exceeded (depends on system configuration).
502 Cannot fetch content/images from the source (timeout, blocked access, or fetch errors).

Suggested Workflow

  1. Call paid/website/crawl with mode=web to get a list of links.
  2. Loop over each link, call paid/website/crawl with mode=article to get the main content + embedded images/videos.