📖Get Crawl

Check the crawl status to see when the crawl completes.

HTTP

POST /docs/get_crawl HTTP/1.1
Api-Key: my_api_key
Content-Type: application/json
Host: my_account_id.us-west-2.aws.chatbees.ai

{
  "namespace_name": "string",
  "collection_name": "string",
  "crawl_id": "string"
}

Response:
{
  "root_url": "string",
  "created_on": int,
  "max_pages": int,
  // crawl_status: 1 = RUNNING, 2 = SUCCEEDED, 3 = FAILED
  "crawl_status": int,
  "crawl_result": {
    // error_code and error_msg is Optional, only set if crawling url1 fails
    "url1": {"char_count": int, "error_code": "string" or null, "error_msg": "string" or null},
    ...
  }
}

Python

import chatbees as cb

# Configure API key
cb.init(api_key="my_api_key", account_id="my_account_id")

col = cb.collection('llm_research')

# check the crawl status
max_waits = 100
waits = 0 
while waits < max_waits:
    status, pages = col.get_crawl(crawl_id)
    if status != cb.CrawlStatus.RUNNING:
        break
    time.sleep(5)