Site-to-markdown knowledge snapshot
Every Sunday night, Firecrawl crawls the site sections you choose and saves each page as clean markdown in My Files — a fresh, agent-readable knowledge base by Monday morning.
Every Sunday night, refresh my markdown knowledge base with Firecrawl. Sections to snapshot: - <https://yoursite.com/docs — our product docs> - <https://yoursite.com/blog — our published posts> - <add more sections: help center, pricing, key landing pages> 1. Crawl each section with Firecrawl and get every page as clean markdown. Cap the total run at 30 pages so credit spend stays predictable; if a section is bigger, prefer its most recently updated pages. 2. Save each page into My Files under knowledge-base/<site-section>/<page-slug>.md, replacing last week's copy. Keep an index file (knowledge-base/INDEX.md) listing every saved page with its URL and a one-line description. 3. Reply with a short changelog: pages added, updated, or gone since last week's snapshot — not the content itself. If nothing changed, one line is enough.
Knowledge base refreshed — Sun 28 Jun 2026, 10:04 PM 29 pages snapshotted → My Files/knowledge-base/ Changes vs last week • Added: docs/webhooks-v2.md (new page) • Updated: docs/rate-limits.md, pricing.md (Pro tier copy changed) • Removed: blog/2024-beta-announcement.md (404s now — deleted the stale copy) • 25 pages unchanged INDEX.md updated — 29 entries.
Your agent is only as current as what it can read. Docs live on the website, the website changes, and the copy your routines and chats reason over quietly goes stale. This routine keeps a living markdown mirror: every Sunday night Firecrawl crawls the sections you chose, and by Monday every page sits in My Files as clean markdown — current, readable, and shared with every routine and chat you run.
How it works
- Firecrawl returns LLM-ready markdown, not raw HTML — so what lands in My Files is exactly
- what an agent reads well.
- My Files is shared across all your routines and chat: snapshot once, and your
- question-answering, drafting, and reporting routines all read the same fresh copy.
- An INDEX.md keeps the base navigable, and the run reports only the diff — added, updated,
- removed — so you can skim what moved.
- Capped at 30 pages per run to keep weekly credit spend predictable.
Make it yours
- Snapshot someone else's site instead: a vendor's API docs, a regulator's guidance, an open
- standard — anything your work depends on reading correctly.
- Pair it with a chat workflow: ask questions Monday morning against the fresh knowledge-base
- folder.
- Raise the page cap (and your Firecrawl plan) for bigger doc sites, or split sections across
- multiple routines.
Run “Site-to-markdown knowledge snapshot” on autopilot
Copy it to your account, tweak the details, and it runs sundays at 10:00 pm.
Use this routine →More Productivity templates
Every Friday, roll up the week's Airtable changes and email a clean summary to your team.
A Monday email of every overdue and due-this-week Asana task, grouped by owner.
Every weekday at 7am, create a fresh, dated standup note in Craft — pre-structured with Yesterday / Today / Blockers, and primed with your calendar for the day.
Each evening, a prep brief for tomorrow's meetings — context and last email thread per attendee.