Speed Signs vs Speed Bumps

Biking my kids home from summer camp the other day, I hit a speed bump a bit too fast. Nothing bad happened, the kids just got a bit startled.

My oldest asked: "Why do they even make speed bumps? Why can't they just put a sign up saying 'Slow down'"?

Her brother knew: "Because people don't listen to signs!"

And that is "guardrails" versus "harness" in a nutshell. No amount of telling an LLM-based agent what to always or never do will give a 100% guarantee. Either the context decays (too much information, so the important instructions get lost among the other stuff) or the guardrail conflicts with something the agent really wants to do. (That one happens a lot with good coding practices that go against mainstream coding practices...)

Whatever you put into an AGENTS.MD file or similar is just like that speed sign. It's optional. To get to the "speed bump" level, the entire way the agent interacts with its environment needs to be designed so that disobeying the rule is impossible. That means controlling which tools can be used in what way, or allowing all tools but only inside a locked-down environment (called a sandbox).

It'll depend on the particular use-case for the agent (and why I'm not sold on a one-fits-all agent like OpenClaw) but in any case, don't rely on signs, rely on speed bumps instead.