The Hidden Rules Behind Claude: How AI System Prompts Quietly Shape What You Get

Every time you type a question into Claude, ChatGPT or Gemini, you're only writing half of the conversation. The other half — the part you never see — is called the system prompt. It's the block of instructions the developer feeds the model before you ever say hello, and it does more to shape the answers you get than most users realise.

System prompts have quietly become one of the most important — and least understood — control surfaces in modern AI. They decide whether a chatbot will talk to you about suicide, how it hedges medical advice, whether it flatters you or pushes back, and even whether it thinks it's a helpful assistant or a project manager. As Australians increasingly lean on tools like Claude for work, study and personal support, understanding these invisible rulebooks isn't just a nerdy curiosity. It's a literacy issue.

What a system prompt actually is

Think of a large language model as an extremely well-read graduate who has just walked into a new job and been handed a one-page brief. The brief says things like: "You work for this company. Be polite. Don't discuss competitors. If someone asks about legal matters, tell them to see a lawyer. Never claim to be human." That brief is the system prompt.

The model doesn't have hard-coded personality. It has weights — statistical patterns learned from training — and then a wrapper of instructions that tells it how to behave in a specific product. A Washington Post analysis put it neatly: these are the hidden rules behind AI, and once you see them, you can even rewrite the way a chatbot talks to you.

Anthropic, the company behind Claude, has been unusually transparent about publishing its system prompts. That transparency has given researchers and journalists a rare window into how safety guardrails are actually implemented — not as mysterious algorithms, but as plain English instructions.

The mental health example: safety written in sentences

One of the clearest illustrations comes from Claude's handling of mental health conversations. In a detailed Forbes analysis, AI commentator Lance Eliot walked through the specific instructions Anthropic uses to steer Claude when a user brings up topics like self-harm, suicidal ideation or acute distress.

The prompt doesn't just say "be careful." It tells the model to watch for signs of crisis, to respond with warmth rather than clinical detachment, to avoid diagnosing, to encourage professional help, and — crucially — not to refuse the conversation outright, because a cold refusal can itself be harmful. In other words, some of the most consequential decisions in AI safety today are being made not by tuning neural networks but by writing careful English sentences.

That's a striking thought. For Australian users who might turn to Claude late at night when Lifeline feels like too big a step, the difference between a helpful response and a robotic "I can't discuss that" is a paragraph of text an engineer wrote somewhere in San Francisco.

The same tool, wildly different behaviour

System prompts don't just handle edge cases. They can transform what an AI is. How-To Geek recently published a walkthrough in which a writer turned Claude into a fully-functioning project management system using nothing but a carefully crafted prompt. No plugins, no new software — just instructions telling Claude to track tasks, assign priorities, follow up on deadlines and format outputs in a specific way.

That's the flip side of the safety story. The same lever that makes Claude gentle around mental health topics also lets a small business owner in Brisbane spin up a bespoke assistant in an afternoon. System prompts are the closest thing modern AI has to programming without code.

And that democratisation is genuinely useful. But it also means the assistant your accountant uses, the one your kid's school pilots, and the one your health insurer trials are all technically the same model wearing different costumes. The behaviour you see depends entirely on the brief someone wrote — a brief you almost never get to read.

Why developers are being told to burn their own prompts

Here's where things get interesting. Boris Cherny, the creator of Anthropic's Claude Code tool, recently urged developers to "sabotage themselves" by deleting their system prompts every six months and starting from scratch.

Why? Because prompts accumulate cruft. Each time a model misbehaves in an unusual way, the natural fix is to add a line: "Don't do X." Then another: "Also don't do Y." Over months, the prompt becomes a graveyard of patches for problems that newer, smarter models have already outgrown. The result is a bloated set of instructions that can actually make the AI less capable — it's spending its attention obeying rules that no longer need to exist.

The advice is a reminder that system prompts are not "set and forget." They're living documents that need to be maintained like any other piece of critical infrastructure. And for organisations rolling out AI at scale — think Australian banks, universities, government services — that maintenance burden is real. Prompt hygiene is becoming a genuine engineering discipline.

What this means for reliability and safety

Put it all together and a few uncomfortable truths emerge.

Reading between the lines

For everyday users, the practical takeaway isn't to panic. It's to recognise that when a chatbot says something surprisingly warm, surprisingly cautious, or surprisingly evasive, you're not seeing the raw model. You're seeing the model plus a paragraph of instructions written by someone whose priorities may or may not match yours.

The Washington Post's invitation — see the hidden rules, then use them to rewrite the article — is the right instinct. The more literate we become about system prompts, the harder it is for anyone to hide behind a chatbot's persona. "The AI said so" is never really true. Somewhere, a human wrote the brief. The interesting question is always: what did they tell it to do, and what did they forget to say?

Related on Bleen

Sources