Non-deterministic systems need deterministic guardrails
A GenAI feature is a probabilistic component embedded inside a deterministic enterprise system. Treating it like a normal microservice — no evaluation harness, no cost ceiling, no data-boundary review — is how pilots fail to reach production. The patterns below are the checklist we use before anything ships.
- Every model call has a token/cost budget and a timeout fallback.
- Every prompt path is versioned and evaluated like code, not tuned in production.
- Every retrieval source is access-scoped to the requesting user, not the app.
- Every output that reaches a customer has a defined human-escalation path.
Reference architectures
RAG & Hybrid Search
Vector + keyword retrieval, chunking strategy, re-ranking, and citation-grounded generation to reduce hallucination.
LLM Gateway
A single ingress for model calls: routing across providers, PII redaction, rate limiting, and audit logging.
Evaluation Harness
Offline and online evals, golden datasets, and regression gates so prompt or model changes can't silently degrade quality.
Fine-Tuning vs. Prompting
Decision tree for when RAG, prompt engineering, LoRA fine-tuning, or full fine-tuning is the right tool for the job.
Guardrails & Content Policy
Input/output filtering, jailbreak resistance, and policy enforcement aligned to responsible-AI commitments.
Cost & Observability
Token-level tracing, latency budgets, and unit-economics dashboards so GenAI spend is a managed line item.
What this layer is measured against
— Viswa