System Design · AI & Agentic Systems

Enterprise Agentic AI Architecture on AWS

A production-oriented agentic AI architecture combining reasoning, enterprise knowledge, memory, governed tool execution, security controls, and end-to-end observability on AWS.

Open presentation

A new local copy, ready to customize. No signup required.

This architecture represents an enterprise AI assistant that can answer grounded questions and perform approved business actions. The agent can reason about a request, retrieve enterprise knowledge, use memory, select approved tools, interact with business systems, and generate a response while policy and observability controls surround the execution path.

Technologies: Amazon Bedrock, Amazon Bedrock AgentCore, Amazon OpenSearch Service, AWS Lambda, Amazon S3, Amazon Cognito, Amazon DynamoDB, Amazon API Gateway, Amazon CloudWatch
Tags: Agentic AI, AI Agents, RAG, Enterprise AI, Generative AI, Tool Use, Knowledge Retrieval, Guardrails, Observability, AWS

How it works

The user authenticates and sends a request through the AI assistant.
API Gateway routes the request to the agent runtime.
The agent evaluates policy, loads relevant memory and determines the required information or actions.
Amazon Bedrock provides model reasoning while OpenSearch supplies grounded enterprise context.
Approved actions are delegated to the tool execution layer, which interacts with enterprise systems.
Agent and tool activity produces telemetry for monitoring, troubleshooting and security review.

Key design decisions

Separate model reasoning from business-system tool execution.
Treat enterprise knowledge as a retrieval capability rather than putting large business datasets directly into prompts.
Persist agent/session context independently from the model.
Enforce authorization and policy controls around tool execution.
Include observability as part of the architecture from the beginning.

Knowledge ingestion

Enterprise documents are stored separately from the online agent path. An ingestion workflow prepares new or updated content and indexes it for retrieval, allowing the agent to obtain relevant enterprise context at runtime.

Security and governance

Ideas for your remix

Replace OpenSearch with another retrieval/vector platform.
Add human approval before high-impact actions.
Split the agent into specialized agents.
Replace AWS services with Azure, Google Cloud or platform-neutral equivalents.
Add asynchronous workflows for long-running tool actions.
Add evaluation, quality monitoring or dedicated audit storage.

Architecture source

aal system-design "0.2"
dictionary "0.2.0"
design EnterpriseAgenticAI "Enterprise Agentic AI Architecture on AWS" {
  element Experience "Experience / Access" {
    type generic.application
    icon "glyph/app-window"
    element WebApp "AI Assistant Web App" {
      type generic.application
      description "Authenticated interface for grounded answers and approved business actions."
    }
    element Cognito "Amazon Cognito" {
      type aws.cognito
    }
    element Gateway "Amazon API Gateway" {
      type aws.api_gateway
    }
  }
  element BusinessUser "Business User" {
    type generic.user
  }
  element AgentPlatform "Agentic AI Platform" {
    type aws.aws_cloud
    element AgentCore "Amazon Bedrock AgentCore" {
      type aws.amazon_bedrock_agentcore
      description "Agent runtime coordinates reasoning, retrieval, session context and approved tools."
    }
    element Bedrock "Amazon Bedrock" {
      type aws.bedrock
      description "Model inference, reasoning and embedding processing."
    }
    element Guardrails "Policy / Guardrails" {
      type generic.policy
      description "Identity-aware policy checks, input/output validation, tool authorization and approval requirements."
    }
    element Memory "DynamoDB — Agent / Session Memory" {
      type aws.dynamodb
      description "Agent and session context stored independently from model inference."
    }
    element ToolExecutor "Lambda — Governed Tool Executor" {
      type aws.lambda
      description "Validates approved tool requests, enforces authorization and invokes enterprise business APIs."
    }
  }
  element Knowledge "Enterprise Knowledge" {
    type generic.storage
    icon "glyph/files"
    element Documents "S3 — Enterprise Documents" {
      type aws.s3.bucket
    }
    element Ingestion "Lambda — Document Ingestion" {
      type aws.lambda
      description "Prepares new or changed documents, obtains embeddings and updates the retrieval index."
    }
    element Search "OpenSearch — Knowledge Index" {
      type aws.opensearch
      description "Retrieval index supplies relevant enterprise context for grounded responses."
    }
  }
  element EnterpriseSystems "Enterprise Systems" {
    type generic.pipeline
    icon "glyph/app-window"
    element BusinessSystems "Enterprise APIs / Business Systems" {
      type generic.external_system
      external true
      description "Business systems exposed through approved, authorized APIs."
    }
  }
  element CloudWatch "Amazon CloudWatch" {
    type aws.cloudwatch
    description "Supporting observability for agent and tool activity, troubleshooting and security review."
  }
  relation UserRequest BusinessUser -> WebApp request_response "Ask / act"
  relation Authenticate WebApp -> Cognito request_response "Authenticate"
  relation AssistantRequest WebApp -> Gateway request_response "Request"
  relation AgentRequest Gateway -> AgentCore invoke "Run agent"
  relation CheckPolicy AgentCore -> Guardrails request_response "Check policy"
  relation Reasoning AgentCore -> Bedrock invoke "Reason"
  relation Retrieval AgentCore -> Search read "Retrieve"
  relation SessionMemory AgentCore -> Memory read "Load context"
  relation ExecuteTool AgentCore -> ToolExecutor invoke "Approved tool"
  relation BusinessAction ToolExecutor -> BusinessSystems request_response "Business action"
  relation NewDocuments Documents -> Ingestion invoke "New / updated"
  relation Embeddings Ingestion -> Bedrock invoke "Embed"
  relation IndexDocuments Ingestion -> Search write "Index"
  relation AgentTelemetry AgentCore -> CloudWatch telemetry "Agent telemetry"
  relation ToolTelemetry ToolExecutor -> CloudWatch telemetry "Tool telemetry"
}