Prompt Engineering

Concept#

Prompt engineering for an application (as opposed to one-off chatting) is really just the discipline of controlling three separate inputs to client.messages.create(), system, messages, and output_config, with enough precision that the model's behavior is predictable across thousands of different end-user inputs, not just the one example you tested by hand.

System prompt vs. user messages#

The system parameter is where persistent, request-independent behavior lives, the model's persona, constraints, and output-format rules that should apply to every turn of a conversation. messages is the actual back-and-forth content, what changes every turn.

const response = await client.messages.create({
  model: "claude-opus-4-8",
  max_tokens: 1024,
  system:
    "You are a support assistant for a SaaS billing product. " +
    "Answer only questions about billing, invoices, and subscriptions. " +
    "If asked about anything else, say so and redirect the user to general support. " +
    "Keep responses under 120 words.",
  messages: [{ role: "user", content: "Why was I charged twice this month?" }],
});

A common beginner mistake is cramming everything, persona, constraints, AND the user's actual question, into a single system string that gets rebuilt from scratch on every request. That works, but it throws away prompt caching (below) and makes debugging harder, because there's no clean separation between "what should never change" and "what's different this turn."

Mid-conversation system messages, the current pattern for injected context#

Older patterns for injecting runtime context mid-conversation (a mode switch, a freshly-fetched fact) relied on stuffing a reminder into a user-turn text block. The current, more precise mechanism on supporting models is a role: "system" entry appended directly into messages, rather than editing the top-level system string:

const response = await client.messages.create({
  model: "claude-opus-4-8",
  max_tokens: 1024,
  system: [
    { type: "text", text: STABLE_SYSTEM_PROMPT, cache_control: { type: "ephemeral" } },
  ],
  messages: [
    ...priorHistory,
    { role: "user", content: "What's my order status?" },
    { role: "system", content: "The user's account was just flagged for a billing dispute, mention this if relevant." },
  ],
});

This matters for two reasons: it leaves the top-level system string byte-for-byte unchanged (so the prompt cache built on it is not invalidated, see below), and it carries operator authority rather than being spoofable text a user could have typed themselves, since it arrives as its own distinct role rather than as text embedded inside a user turn.

Prompt caching, a prefix match, not a semantic cache#

cache_control does not mean "cache similar requests." It caches an exact byte-for-byte prefix of the rendered request. Any change anywhere in that prefix, a different timestamp, a reordered JSON key, a different tool list, invalidates the cache for everything after that point.

const response = await client.messages.create({
  model: "claude-opus-4-8",
  max_tokens: 1024,
  system: [
    {
      type: "text",
      text: LARGE_SHARED_KNOWLEDGE_BASE_TEXT, // e.g. 20K tokens of product docs
      cache_control: { type: "ephemeral" }, // 5-minute default TTL
    },
  ],
  messages: [{ role: "user", content: "How do I set up SSO?" }],
});
 
// response.usage.cache_read_input_tokens, served from cache, ~0.1x cost
// response.usage.cache_creation_input_tokens, written to cache this request, ~1.25x cost
// response.usage.input_tokens, full-price, uncached tokens

The single most common way teams accidentally disable caching entirely is interpolating something that changes every request, new Date(), a request UUID, a user id, directly into the system prompt text. That one interpolation sits early in the byte sequence and invalidates every cache read for the rest of the request, even if 95% of the system prompt is otherwise identical every time. The fix is always the same: keep the frozen, shared portion first (with the cache_control breakpoint at its end), and put anything that varies per-request or per-user after that breakpoint, in messages instead.

Replacing assistant-turn prefills with structured outputs#

An older prompting trick forced the response into a particular shape by pre-filling the start of the assistant's turn (e.g. ending your messages array with { role: "assistant", content: '{"name": "' } to force JSON). This is deprecated on current-generation models and returns a 400 error. The current, more robust replacement is output_config.format:

const response = await client.messages.create({
  model: "claude-opus-4-8",
  max_tokens: 512,
  output_config: {
    format: {
      type: "json_schema",
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          plan: { type: "string", enum: ["free", "pro", "enterprise"] },
        },
        required: ["name", "plan"],
        additionalProperties: false,
      },
    },
  },
  messages: [{ role: "user", content: "Jane Doe is on the Pro plan."

Structured outputs guarantee the response actually validates against your schema, a stronger guarantee than a prefill ever gave you, since a prefill only nudged the model's starting tokens without enforcing the rest of the structure.

Try It#

Predict which of these two system prompts produces more consistent behavior across many different user inputs, before checking the solution.

// Prompt A
system: "Be helpful and answer questions about our product."
 
// Prompt B
system: "You are a support assistant for Acme's billing product. " +
  "Scope: billing, invoices, subscriptions only. " +
  "If the question is out of scope, say: 'I can only help with billing questions, " +
  "for anything else, contact support@acme.com.' " +
  "Response length: under 100 words, no markdown headers."

Which one is more likely to produce predictable, on-brand behavior at scale, and why?

Implement It Yourself#

Build a tiny prompt-template helper that enforces the separation between stable, cacheable system content and per-request variable content, the actual discipline that makes caching work in practice.

interface PromptTemplate {
  stableSystem: string; // never changes across calls, safe to cache
  buildMessages: (userInput: string, context?: string) => Anthropic.MessageParam[];
}
 
function createSupportPrompt(): PromptTemplate {
  const stableSystem =
    "You are a support assistant for Acme's billing product. " +
    "Scope: billing, invoices, subscriptions only. Response length: under 100 words.";
 
  return {
    stableSystem,
    buildMessages(userInput, context) {
      const messages: Anthropic.MessageParam

This is the actual mechanism real prompt-management systems use: a fixed, versioned, cacheable "core" prompt plus a clearly separated channel for runtime-varying context, never one giant string rebuilt from scratch per request.

Under the Hood#

Prompt caching's prefix-match behavior is a direct consequence of how the request is serialized and hashed before being sent, reordering a JSON object's keys or changing a single character anywhere in the prefix produces different bytes, the same non-negotiable rule covered generally in Generics in TypeScript around structural type matching being exact, not fuzzy. The mid-conversation role: "system" message pattern exists specifically because text embedded in a user-role turn is otherwise indistinguishable from anything the end user typed, which is the same class of trust-boundary problem covered in Cross-Site Scripting (XSS), untrusted input and trusted instructions must stay in distinguishable channels, not collapsed into one string.

Common Mistakes#

1. Interpolating a timestamp or request ID into the system prompt#

// ❌ invalidates prompt caching on every single request
system: `Current time: ${new Date().toISOString()}\n\nYou are a helpful assistant...`

Every request now has a unique prefix, so cache_read_input_tokens will be zero forever, even though the actual instructions never change. Move genuinely dynamic values (like a timestamp) into messages, after the cache breakpoint, or drop them if they aren't load-bearing for the model's behavior.

2. Cramming the user's actual question into the system prompt#

// ❌ system prompt now changes every request, the whole point of `system` is lost
system: `You are a helpful assistant. The user asked: "${userQuestion}"`

This defeats caching (system changes every call) and blurs the model's understanding of what's an instruction versus what's user content. The user's question belongs in messages as a user-role turn; system should describe stable behavior only.

3. Relying on an assistant-turn prefill for structured output#

// ❌ 400 error on current-generation models
messages: [
  { role: "user", content: "Extract the name and plan." },
  { role: "assistant", content: '{"name": "' },
]

Assistant-turn prefills are deprecated on current-generation models and now return a 400 error. Use output_config: { format: { type: "json_schema", schema: {...} } } instead, it's both supported and gives you an actual validation guarantee the prefill trick never did.

Best Practices#

• Keep system for stable, request-independent behavior only, persona, scope, format rules. Put anything that changes per-request in messages.
• Put a cache_control breakpoint at the end of the stable portion of system (or the last shared block of a long messages prefix) so repeated requests actually hit the cache.
• Use a role: "system" message in messages for context that needs to arrive mid-conversation, instead of editing the top-level system string or embedding it in a user turn.
• for anything that needs a guaranteed shape (JSON, an enum-constrained field).

Performance Tips#

• Verify caching is actually working by checking response.usage.cache_read_input_tokens on repeated requests, if it's zero, something in the prefix is varying and needs to be found and moved.
• A cache breakpoint requires a minimum prefix length (model-dependent, roughly 1024, 4096 tokens), a short system prompt below that threshold silently won't cache at all, with no error.
• Prompt caching's write cost (~1.25x for the default 5-minute TTL) only pays off after roughly two requests reuse the same prefix, don't bother caching a system prompt that's only ever sent once.