Who Holds the Keys?

Who Holds the Keys?

We keep wiring LLMs into everything, and we keep making the same mistake: we hand them the keys.

A token in the system prompt. A secret in the context. It works, until someone writes one clever line and the model repeats the token for them. Then you learn that a language model can be very smart and very naive at the same time.

The fix is not a better prompt. It is keeping identity out of the model. The user logs in at the gateway. The backend attaches their token to the HTTP header when the model asks for a tool. The server checks the role, runs what is allowed, and returns a 403 for the rest.

The model never sees the token. It only sees a tool that worked, or one that failed. All the real access control lives below, in plain code the model cannot read or rewrite.

So when I design these systems, there is one question I keep asking: who actually holds the keys? If the answer is the model, it is the wrong design.