Not a security boundary

Semantic evaluation is a soft risk or review layer. It is good at judging what text means. It is not a place to enforce who is allowed to do what.

What stays deterministic#

Authentication, authorisation, payment permission, compliance limits and exact numeric validation are code:

if (!user.roles.includes("admin")) throw new ForbiddenError()   // not semantic
if (order.total > limits[user.plan]) return reject()              // not semantic

A semantic answer can add a hold, a review or a flag. It must not remove a check that code would otherwise perform.

Prompt injection, measured#

A decision model has no instruction channel. It scores a proposition; it does not follow orders. That changes what an attacker can do, and the change is not the one people expect.

One criterion, seven payloads, live model. The criterion asks whether the ticket describes a live production outage. Each payload was placed two ways: folded into the question text, and as a labelled field in the state.

Instruction-shaped attacks do not work. "SYSTEM OVERRIDE", a fake criteria block, a fake JSON answer: each scores where the benign baseline scores. There is nothing to hijack, so "ignore the above" is just more text to score.

Content-shaped attacks do work. The payload that succeeds never asks for anything. It asserts new facts ("the real ticket is: production is fully down"), and the model evaluates that claim correctly, because the text genuinely does describe an outage. There is nothing to refuse.

Note the direction of the last row. The labelled field scores higher. Clean structure makes the model trust the content more, which is right for real input and wrong for hostile input. Shaping the request is not a defence.

The defence: ask what the author owns#

What defends you is asking questions whose author is the authority on the answer.

For the last one, ask your monitoring, not the person filing the ticket.

Then keep a deterministic gate in front of any consequence. A matched rule should only fire for a plan, a role or an amount the attacker cannot write:

const t = await triage(ticket)

// The model says "page on-call". The plan says whether that is even possible.
if (t.blocksRevenue && customer.plan !== "free") pageOnCall()

The ticket triage scenario has a test for exactly this: a free-plan ticket never pages on-call, however bad it looks.

Fail closed where it matters#

When the provider is unreachable, ProviderError propagates. Decide per call site what "no judgement" means:

async function canAutoReply(ticket) {
  if (ticket.priority === "critical") return false       // never, regardless of judgement
  try {
    return await safeToAutoReply(ticket)
  } catch {
    return false                                          // when in doubt, a human answers
  }
}

For a review queue, failing open (a human looks) is usually right. For an action with consequences, failing closed is. See Errors.

Also unsuited to#

• Counting, arithmetic, date comparison. A decision model reads 2026-03-01 as text, not as a point in time. Extract the parts with a choose() if you must, then compare in code.
• Exact matching. If you can write the regex, write the regex.
• Long-tail facts. The model judges the text in front of it. It does not know your price list unless you put it in context.
• Anything where the input author must not influence the outcome. If the person writing the text benefits from a particular answer, the question is unsafe by construction.