📖Crawl Operations

Chatting for the contents on a website is very simple. Just use our 3 APIs:

• Create Crawl: start crawling a website.

• Get Crawl: get the crawl results, including the crawl status, crawled pages, etc.

• Index Crawl: after crawl completes and checks the crawl results, index the crawl results into the collection.

• Delete Crawl: delete all crawled pages from the collection.

If you re-crawl the website, please make sure to call delete_crawl API before calling the index_crawl API. Or else, the same page may be indexed twice in the collection. Enhancements are planned to support incremental crawl.