Results for “crawler”

markdown-crawler

paulpierre/markdown-crawler

A multithreaded web crawler that recursively crawls a website and creates markdown files for each page.

Scrapegraph-ai

ScrapeGraphAI/Scrapegraph-ai

A Python scraping library with LLM and direct graph logic

trafilatura

adbar/trafilatura

Python & Command-line tool to gather web text and metadata

chatWeb

SkywalkerDarren/chatWeb

Crawls web pages and documents, extracts text content, generates embeddings for summaries and answers questions.