@vavvolo.bsky.social
Reposted
Some other great resources in this space:
robotstxt.com/ai
github.com/ai-robots-tx...
I didn't whole-sale copy those above, as they block not only *training*, but also *inference*.
But if an agent wants to read my site on behalf of a user to answer a question, I'm fine with that.
robotstxt.com/ai
github.com/ai-robots-tx...
I didn't whole-sale copy those above, as they block not only *training*, but also *inference*.
But if an agent wants to read my site on behalf of a user to answer a question, I'm fine with that.
AI / LLM User-Agents: Blocking Guide
Find out how to block your content from being used for AI/LLM training with robots.txt. Created by ex-Google engineer Fili.
robotstxt.com
January 10, 2026 at 6:43 PM
Some other great resources in this space:
robotstxt.com/ai
github.com/ai-robots-tx...
I didn't whole-sale copy those above, as they block not only *training*, but also *inference*.
But if an agent wants to read my site on behalf of a user to answer a question, I'm fine with that.
robotstxt.com/ai
github.com/ai-robots-tx...
I didn't whole-sale copy those above, as they block not only *training*, but also *inference*.
But if an agent wants to read my site on behalf of a user to answer a question, I'm fine with that.
this article has interesting recommendations about structuring agent md files www.humanlayer.dev/blog/writing...
Writing a good CLAUDE.md
`CLAUDE.md` is a high-leverage configuration point for Claude Code. Learning how to write a good `CLAUDE.md` (or `AGENTS.md`) is a key skill for agent-enabled software engineering.
www.humanlayer.dev
December 8, 2025 at 12:25 PM
this article has interesting recommendations about structuring agent md files www.humanlayer.dev/blog/writing...