How to Build Reliable AI Agents with NestJS: Production Architecture

An AI agent demo can be a prompt, a model, and one tool call. A production AI agent is a software system: it needs an API boundary, validated tools, durable state, permissions, retries, observability, and a human fallback.

That distinction matters for businesses. The model may decide what to do next, but ordinary backend engineering still determines whether the result is safe, traceable, and useful.

This guide explains how I would structure AI agents with NestJS for production. It connects the modular patterns from my NestJS backend architecture guide with the controls an agent needs when it can read data, call APIs, or change a business system.

The goal is not maximum autonomy. The goal is the smallest amount of model-driven decision-making that reliably improves a real workflow.

First decide: automation, AI workflow, or agent?

These terms are often mixed together, but they describe different control models.

• Automation: fixed rules decide every step. Example: when a form is submitted, create a CRM record and notify sales.
• AI workflow: a controlled process contains one or more AI steps. Example: classify the lead and summarize its request before the normal workflow continues.
• AI agent: the model can select tools or choose the next action within defined limits. Example: inspect a support request, query approved systems, ask for missing information, and draft a resolution.

Use fixed automation when the inputs and decisions are predictable. Add AI when classification, extraction, summarization, or flexible language creates measurable value. Add agent behavior only when changing steps and tool selection are genuinely necessary.

This decision prevents a common architecture mistake: placing the model in charge of a process that should have remained deterministic.

A production NestJS agent architecture

A reliable design separates six responsibilities:

1. API layer: authenticates the caller, validates input, and returns a job or run identifier.
2. Workflow layer: owns the business process and decides where model reasoning is allowed.
3. Agent runtime: sends instructions and context to the model and receives structured actions.
4. Tool layer: exposes narrow, validated business capabilities.
5. State and queue layer: persists progress, handles retries, and prevents duplicate side effects.
6. Observability and approval layer: records decisions, costs, failures, and pending human actions.

NestJS fits this architecture because modules and dependency injection make the boundaries explicit. The agent runtime becomes one provider inside a normal backend—not the entire backend.

Client
  -> NestJS API
  -> Workflow service
  -> Queue
  -> Agent worker
       -> Model
       -> Approved tools
       -> Human approval when required
  -> PostgreSQL audit/state
  -> Result or notification

1. Keep the HTTP request short

Agent runs can involve several model and tool calls. Do not keep a public HTTP request open while a long workflow runs.

The controller should validate the request, create a run record, enqueue the work, and return 202 Accepted.

@Post("lead-triage")
@HttpCode(HttpStatus.ACCEPTED)
async startLeadTriage(
  @Body() input: CreateLeadTriageDto,
  @CurrentUser() user: AuthenticatedUser,
) {
  const run = await this.agentRuns.create({
    type: "lead-triage",
    requestedBy: user.id,
    input,
  });

  await this.agentQueue.add(
    "lead-triage",
    { runId: run.id },
    {
      jobId: run.id,
      attempts: 3,
      backoff: { type: "exponential", delay: 2_000 },
      removeOnComplete: 100,
    },
  );

  return { runId: run.id, status: "queued" };
}

The stable jobId helps stop the same request from producing duplicate jobs. Authentication, rate limits, request-size limits, and DTO validation still belong at the API boundary.

For sensitive endpoints, combine this with the Redis protections in my API rate-limiting guide.

2. Put business rules outside the prompt

A prompt should explain the task and available tools. It should not be the only place where your company’s rules exist.

Rules such as these belong in code or configuration:

• which tenant owns a record;
• which roles can read or change it;
• whether a refund needs approval;
• the maximum amount or scope of an action;
• which fields may be sent to a model;
• how many tool calls a run may perform;
• when the workflow must stop.

This separation makes rules testable and prevents a prompt edit from silently changing authorization.

@Injectable()
export class RefundPolicy {
  evaluate(input: RefundRequest, actor: Actor): PolicyDecision {
    if (!actor.permissions.includes("refund:request")) {
      return { allowed: false, reason: "missing_permission" };
    }

    if (input.amount > 100) {
      return { allowed: false, requiresApproval: true };
    }

    return { allowed: true };
  }
}

The model may recommend an action. The policy provider decides whether the system may execute it.

3. Design narrow tools with strict schemas

An agent tool is an API contract. Give it one clear purpose, validate every argument, and return the minimum data the agent needs.

A tool called runSql is too broad for most business agents. A tool called findCustomerOrders with tenant checks, result limits, and an explicit response shape is safer and easier to observe.

const FindCustomerOrdersSchema = z.object({
  customerId: z.string().uuid(),
  limit: z.number().int().min(1).max(20).default(10),
});

async function findCustomerOrders(
  rawInput: unknown,
  context: AgentContext,
) {
  const input = FindCustomerOrdersSchema.parse(rawInput);

  return ordersRepository.findRecent({
    tenantId: context.tenantId,
    customerId: input.customerId,
    limit: input.limit,
  });
}

For every tool, define:

• input and output schemas;
• authentication and tenant context;
• permission checks;
• timeout and retry behavior;
• idempotency behavior;
• maximum result size;
• whether human approval is required;
• audit fields that must be recorded.

The current OpenAI Agents SDK tool documentation describes agents as models configured with instructions and tools. The important engineering work is defining what those tools are allowed to do.

4. Separate reads from side effects

Read tools and write tools do not carry the same risk.

An agent may be allowed to search a knowledge base or retrieve an order automatically. Sending an email, changing a CRM stage, issuing a refund, deleting a file, or publishing content should pass stronger controls.

Use three execution levels:

• Automatic read: scoped retrieval with logging and result limits.
• Automatic low-risk write: idempotent, reversible action inside a small policy boundary.
• Approval-required write: financial, external, destructive, privacy-sensitive, or difficult-to-reverse action.

The human-in-the-loop guide shows an approval flow that pauses a run and resumes it from stored state. Your NestJS application should also persist the approval request, approver, decision, reason, and expiry.

5. Use queues for durability, not only speed

Queues smooth traffic peaks, move long work away from the Node.js request cycle, and provide retry and lifecycle events. The official NestJS queue guide also notes that Redis-backed jobs persist across process restarts.

For agent workloads, a queue provides:

• controlled concurrency;
• retry policies for temporary provider failures;
• delayed resumption after human approval;
• separate worker scaling;
• dead-letter handling;
• a stable run identifier;
• protection against an expensive burst of model calls.

Retries require care. Never repeat a side effect simply because a worker failed after the external API completed.

Before calling a write tool, create an idempotency key from the run, tool, and intended action. Save the result. A retry can then return the existing result instead of sending the action again.

const idempotencyKey = `${run.id}:send-follow-up:${lead.id}`;

const previous = await toolExecutions.findByKey(idempotencyKey);
if (previous?.status === "completed") return previous.output;

return toolExecutions.executeOnce(idempotencyKey, () =>
  crm.sendFollowUp({ leadId: lead.id, draftId }),
);

6. Persist explicit run state

Chat history is not a complete workflow state.

Store a run record with fields such as:

• run ID, tenant ID, workflow type, and version;
• status and current step;
• sanitized input and structured output;
• tool calls and their results;
• model/provider identifiers;
• token usage, latency, and estimated cost;
• retry count and error category;
• approval status;
• timestamps and retention policy.

This record gives support staff a way to explain what happened. It also lets you resume a paused process without asking the model to reconstruct the truth from a long conversation.

PostgreSQL is a strong default for run and audit data. Redis can support locks, rate limits, short-lived state, and queues. Object storage is better for large documents. Use each system for the responsibility it handles well.

7. Add guardrails at every boundary

Guardrails are not one moderation prompt at the beginning.

Production controls can include:

• input validation before the first model call;
• prompt-injection handling around retrieved content;
• tool input validation before execution;
• output validation after execution;
• sensitive-data redaction;
• allowlists for URLs, files, recipients, and operations;
• tool-call and token budgets;
• loop detection;
• human approval;
• final-output schema validation.

The OpenAI Agents SDK guardrails guide distinguishes input, output, and tool guardrails. Tool guardrails matter because a valid-looking final answer does not prove that every intermediate action was safe.

Treat content retrieved from email, websites, documents, and third-party systems as untrusted data. It may contain instructions, but those instructions do not become system authority.

8. Observe outcomes, not just exceptions

A run can complete without throwing an exception and still produce a poor business result.

Track technical metrics:

• model and tool latency;
• error and retry rate;
• queue age;
• token usage and cost;
• tool-call count;
• approval wait time.

Track workflow metrics:

• classification accuracy;
• manual correction rate;
• resolution time;
• duplicate or reversed actions;
• percentage of runs escalated to a human;
• business outcome after the automation.

Use trace IDs across the API request, queue job, model run, and tool call. Never place secrets or unrestricted personal data in logs.

Example: controlled lead-intake agent

Consider a service business receiving leads from a website.

A reliable workflow could:

1. Validate and store the form submission.
2. Check spam and rate-limit rules.
3. Ask the model for a structured category, urgency, and summary.
4. Query an approved service catalogue.
5. Create a CRM record through an idempotent tool.
6. Draft a reply.
7. Require approval for unusual commitments or pricing.
8. Notify the correct owner.
9. Record the final disposition for evaluation.

Most of this is a deterministic workflow. Agent behavior may be useful for deciding which approved information to retrieve or what clarification to request. The agent does not need permission to redesign the entire sales process.

That is what “controlled AI automation” means in practice.

Production checklist

Before releasing an AI agent, confirm:

• The business process and success metric are written down.
• A fixed workflow was considered before agent autonomy.
• Every tool has a strict schema and permission boundary.
• Tenant and user authorization are enforced in code.
• Long runs execute in a queue.
• Side effects are idempotent.
• Run state survives a restart.
• High-impact actions require approval.
• Tool calls, costs, latency, and outcomes are observable.
• Timeouts, retries, and dead-letter handling are tested.
• Sensitive data has an explicit retention policy.
• There is a manual fallback and a way to disable the agent.

The practical takeaway

NestJS gives AI agent systems a disciplined backend structure: modules for business domains, injectable providers for model and tool adapters, DTO validation at boundaries, guards for authorization, and queues for durable execution.

The model is one component. Reliability comes from the system around it.

If you are deciding whether a process needs fixed automation, a controlled AI workflow, or a tool-using agent, see my AI automation and custom software services or review the backend and software projects I have built.