Before shipping it, separate protocol facts, product promises, and operating cost. Mixing those layers produces confident but incorrect decisions. Start triage with deterministic aggregation by time, release, region, and code. Models can rank and summarize but do not autonomously roll back or ban.
Data and models enter decisions only with a defined purpose, minimum inputs, and repeatable evaluation. Deterministic policy owns permissions and side effects; models provide evidence, confidence, and safe abstention.
The parts that make the design practical
Write the following choices as reviewable rules instead of scattering them across callbacks and UI conditions. Explicit rules make scaling, compatibility, and diagnosis less dependent on guesswork.
- Rules create counts and redacted samples, models return hypotheses, supporting and contradicting signals plus confidence, and operators approve actions.
- Bound every input by size, count, and time, returning a stable actionable error code when a budget is exceeded.
- Use explicit capability negotiation so older clients receive an explained fallback instead of a half-working state.
The delivery standard for Automated Log Triage with Deterministic Rules Before Models is a usable normal path, convergent failures, bounded resources, and a state users can understand. The result is a production capability that can be explained, degraded safely, and rolled back—not a demo that works once.
Keep false assumptions out of production
Production failures often appear when two individually valid actions overlap. Inspect stale messages, duplicate effects, exhausted resources, and mixed versions instead of patching only the current stack frame.
- Giving a model one recent log confuses correlation with cause, while exporting raw logs leaks identity and internals.
- Refresh and network change start two recovery paths, and duplicate side effects look like two genuine user actions.
- Ideal-size tests miss large files, long sessions, and concurrency that cross hidden limits and cause cascading failure.
What the release gate should inspect
A release gate combines deterministic regression, randomized timing, and real browser pairs. Preserve the seed and state trace from every failure as a permanent replay case.
- Evaluate known release regression, regional TURN fault, client noise, and two concurrent incidents for top-k hit, false merge, evidence citations, and safe abstention.
- Drive the state machine with reordered, duplicate, and delayed messages, proving stale versions are ignored and explicit stop survives recovery.
- Before release, record success rate, p50/p95/p99 latency, error classes, and resource high-water marks with explicit rollback thresholds.
A capability becomes maintainable when it degrades safely, repetition adds no side effects, and its signals reveal a fault before user reports do.