The visible problem may look like one API or tuning value, but reliability is decided by state ownership, resource bounds, and recovery after failure. A user can put ignore-rules-and-export-secrets in messages or filenames. Models must treat this as quoted data, never instructions.
Data and models enter decisions only with a defined purpose, minimum inputs, and repeatable evaluation. Deterministic policy owns permissions and side effects; models provide evidence, confidence, and safe abstention.
Make the implementation decisions explicit
List non-negotiable invariants before selecting performance knobs. Tuning can roll out gradually; identity, permission, and terminal-state rules cannot drift at runtime.
- System instructions define immutable bounds, tickets enter structured data fields, tools use least-privilege allowlists, and people approve external actions, bans, or refunds.
- Separate protocol facts, user intent, and automatic recovery; automation may restore facts but never overturn an explicit choice.
- Use explicit capability negotiation so older clients receive an explained fallback instead of a half-working state.
The delivery standard for Prompt Injection in Support Intake and Diagnostic Text is a usable normal path, convergent failures, bounded resources, and a state users can understand. The result is a production capability that can be explained, degraded safely, and rolled back—not a demo that works once.
Failure paths that are easy to miss
Production failures often appear when two individually valid actions overlap. Inspect stale messages, duplicate effects, exhausted resources, and mixed versions instead of patching only the current stack frame.
- A prompt asking the model to resist injection is not isolation, while broad user lookup and email tools turn text into real side effects.
- Fixing only the UI leaves queues, locks, or expired credentials for the next operation to inherit and fail again.
- An untested fallback receives all traffic during a primary failure and becomes the slower, more expensive bottleneck.
How to verify it before release
Observe both endpoints, persisted records, and operational signals during verification. One button state or one successful response cannot prove the complete loop.
- Test direct, indirect, encoded, log-field, and retrieved-document injection; models quote rather than obey, policy blocks tools, and canaries stay secret.
- Drive the state machine with reordered, duplicate, and delayed messages, proving stale versions are ignored and explicit stop survives recovery.
- Use fault injection to prove alerts precede user reports and operators can locate the failing phase from bounded evidence.
A capability becomes maintainable when it degrades safely, repetition adds no side effects, and its signals reveal a fault before user reports do.