Governed AI Gateway · AWS Reference Architecture

01 · FOUNDATION

Four decisions. Four owners.

The platform remains governable when authentication, authorization, content rights and AI safety are modeled as separate decisions with independent evidence.

Sources S01–S03

Authentication

Question: Who is making the request?

Enterprise OIDC. Durable subject ID. Tenant. Session assurance. Authentication time.

Authorization

Question: May this principal perform this platform action?

Amazon Verified Permissions. Cedar policies. Default-deny. Explicit action and resource.

Content rights

Question: May this principal use this asset now, here and for this purpose?

Entitlement, subscription, time window, country, classification and derivative-use terms.

AI safety

Question: Is this input, retrieved context, action or output acceptable?

Guardrails, schema validation, sanitizers, prompt-injection defenses and evaluations.

Models generate content. They never authorize access. Primary security invariant

Fail closed

Deny when identity, tenant, rights, policy version, geography or required safety enforcement cannot be established.

No direct provider access

Applications do not receive broad model invocation permissions. The governed gateway is the approved invocation boundary.

02 · SYSTEM TOPOLOGY

Separate the path from the policy.

The data plane handles low-latency requests. The control plane governs policies, models, prompts, guardrails and evaluations without coupling changes to application code.

Sources S01–S06

Governed AI Gateway topology Edge, data plane, control plane, knowledge, models and evidence

Rendering architecture…

Data plane

Identity, request normalization, authorization, retrieval, provider invocation, output enforcement and evidence.

Control plane

Policy source, schemas, configuration, model catalog, prompts, guardrail versions, evaluations and approvals.

Evidence plane

Operational telemetry and immutable governance events, separated from raw prompt or response retention.

Lambda-first, not Lambda-only

The contract, policy model, audit schema and provider adapters must remain portable to ECS, EKS or another Kubernetes runtime.

03 · TRUST PATH

Authorize before context becomes capability.

Every request follows the same order. Each stage emits enough evidence to explain why the final response was allowed, constrained, blocked or denied.

Sources S03–S05

Request sequence Knowledge answer with independent input, retrieval and output controls

Rendering request path…

Validate edge context

Strip spoofable headers. Verify signature, key ID, issue time, expiry and request binding.

Normalize identity

Convert provider claims into a stable platform principal. Email is not the durable key.

Authorize the action

Evaluate principal, tenant, purpose, subscription, country and application against Cedar policy.

Inspect input

Enforce schema, size, Unicode, secret, PII, content and prompt-injection controls before retrieval.

Authorize retrieved resources

Metadata narrows candidates. Per-resource authorization remains the final boundary.

Invoke through an adapter

Resolve a capability alias to an approved provider, model, region, quota and guardrail version.

Validate output

Recheck safety, structured output, URLs, citations, data leakage and rights-dependent excerpt limits.

Publish evidence

Record decision provenance, content IDs, policy hashes, model route and usage without default raw-content logging.

04 · RIGHTS ENGINE

Entitlement is data, not code.

Premium access, person-specific grants, embargoes, time windows, purpose limits and geographic restrictions should be editable as rights data and policy.

Sources S01, S02, S05

Rights-aware retrieval Retrieval filters improve precision; resource authorization enforces security

Rendering rights model…

Rights dimension	Example attribute	Enforcement
Identity	`allowedPrincipalIds`, groups, tenant	Per-resource Cedar decision
Subscription	`minimumSubscription = premium`	Policy plus entitlement record
Time	`validFrom`, `validUntil`, embargo	Signed request time in context
Geography	Allowed and denied countries or legal regions	Signed edge context, rechecked at origin
Purpose	Support, research, summary, publication	Explicit action and request purpose
AI usage	Retrieval, summarization, transformation, training	Separate rights flags and actions
Output	Maximum excerpt length, citation requirement	Prompt and response post-processing

content-rights.ts

export interface ContentRights {
  contentId: string;
  contentVersion: string;
  tenantId: string;
  classification: "public" | "internal" | "confidential" | "restricted";

  requiredEntitlement?: string;
  minimumSubscription: "free" | "standard" | "premium" | "enterprise";
  allowedPrincipalIds?: string[];
  allowedGroups?: string[];

  allowedCountries?: string[];
  deniedCountries?: string[];
  validFromEpoch?: number;
  validUntilEpoch?: number;
  allowedPurposes: string[];

  aiRetrievalAllowed: boolean;
  summarizationAllowed: boolean;
  derivativeGenerationAllowed: boolean;
  modelTrainingAllowed: boolean;

  maximumExcerptCharacters?: number;
  policyVersion: string;
}

Retrieval filters are not the final authorization boundary

Use filters to reduce the candidate set. Load authoritative metadata and authorize each returned content resource before it enters a prompt.

05 · SAFETY

Guard the whole context path.

Input guardrails alone are insufficient. Retrieved documents, structured tool arguments, generated URLs, image prompts and final output each require separate enforcement.

Sources S03, S04, S10, S11

Before retrieval

Schema and payload limits
Unicode normalization
Secret and PII detection
Direct prompt-injection analysis
Denied topics and abuse controls

Before inference

Authorized context only
Instruction/data separation
Active markup removal
Indirect prompt-injection analysis
Protected prompt template

Before release

Output guardrail
Structured schema validation
URL and HTML sanitization
Citation authorization check
Rights-based excerpt enforcement

Layer	Primary controls	Failure posture
Edge	WAF, bot controls, coarse quotas, size limits	Reject obvious abuse early
Identity	JWT claims, issuer, audience, session assurance	Deny unverified identity
Request	JSON schema, normalization, replay and idempotency	Reject malformed request
Retrieval	Ingestion quarantine, metadata filters, resource authorization	Exclude unauthorized context
Model	Approved alias, token limits, region policy, timeout	No uncontrolled fallback
Output	Guardrail, data loss prevention, citation and format checks	Block, transform or safe refusal
Evidence	Redaction, digests, immutable audit archive	Do not log raw sensitive content by default

Retrieved content is untrusted input

A knowledge document can contain instructions designed to override the system, exfiltrate data or trigger tools. It must never inherit authority from its storage location.

06 · MODEL PORTABILITY

Route capabilities, not model IDs.

Clients request governed capabilities. The platform resolves provider, model, region, quota, guardrail and cost profile at runtime.

Sources S03, S12

Client-facing aliases

knowledge-balanced, regulated-text, low-latency, approved-image.

Aliases remain stable when providers, versions or regions change.

Provider adapter ownership

Credential handling, request translation, streaming normalization, retries, usage extraction and provider error mapping.

Authorization remains outside the adapter.

model-provider-adapter.ts

export type AICapability =
  | "chat"
  | "structured-output"
  | "embeddings"
  | "image-generation"
  | "tool-use";

export interface ModelProviderAdapter {
  readonly providerId: string;

  supports(capability: AICapability): boolean;

  invoke(
    request: GovernedAIRequest,
    context: GovernedContext,
    signal: AbortSignal
  ): AsyncIterable<GatewayEvent>;
}

No silent safety downgrade

A provider outage may trigger an approved route change. It must not remove a required guardrail, broaden data residency or reduce the authorization posture.

07 · CHANGE GOVERNANCE

Policy changes are production changes.

Cedar, AppConfig, prompts, guardrails and model routes require schema validation, regression tests, provenance and staged release—even when no application code changes.

Sources S01–S03

Governance release pipeline Policy-as-code with evaluation gates, approval and automatic rollback

Rendering control plane…

Verified Permissions

Fine-grained action and resource policy. Use a strict Cedar schema and policy validation for production stores.

AWS AppConfig

Model routes, quotas, country controls, feature flags and kill switches with validators, staged deployment and rollback alarms.

Guardrail registry

Named, versioned safety policies. Routes pin a tested guardrail version rather than resolving an unversioned mutable configuration.

repository-shape.txt

ai-governance/
├── cedar/
│   ├── schema.json
│   ├── platform/
│   ├── knowledge/
│   └── images/
├── appconfig/
│   ├── model-routes.json
│   ├── quotas.json
│   └── emergency-controls.json
├── guardrails/
├── prompts/
├── evals/
│   ├── authorization/
│   ├── safety/
│   ├── retrieval/
│   └── model-quality/
└── provenance/
    └── release-manifest.schema.json

08 · ASSURANCE

Every answer needs an evidence trail.

The system must reconstruct who requested what, which rights and policies applied, what content was authorized, which route ran and what safety decisions occurred.

Sources S07–S09

Operational telemetry

CloudWatch metrics and logs, X-Ray traces, latency, errors, quotas, provider health and cost units.

Governance evidence

Structured events through Data Firehose into a dedicated, KMS-encrypted S3 archive protected by Object Lock.

Evidence category	Record	Avoid by default
Identity	Principal hash, tenant, application, session assurance	Email or unnecessary profile data
Authorization	Decision, policy store, bundle hash, determining policies	Opaque “allowed=true” only
Content	Requested, allowed and denied content IDs; rights versions	Full protected document text
AI route	Provider, alias, resolved model, prompt and guardrail versions	Unversioned “latest” references
Usage	Input/output tokens, latency, cost units, result class	Secrets, access tokens and raw credentials
Integrity	Input/output digests and archive checks	Raw prompt/response retention as the standard path

Authorization evaluations

Zero cross-tenant, premium, person, embargo, time-window or geography leakage.

Safety evaluations

Direct and indirect injection, secret extraction, PII, jailbreaks, malicious URLs and tool misuse.

Quality evaluations

Groundedness, citation precision, retrieval recall, refusal correctness, answer relevance and hallucination rate.

Evaluate policy and retrieval changes—not only models

A safer model cannot compensate for an authorization regression or a metadata-filter bug. Release gates must cover the entire trust path.

09 · EDGE

Enrich at the edge. Decide at the origin.

Edge infrastructure can reduce abuse and attach signed geography or network risk. It should not become the sole authority for legal or rights-sensitive access.

Sources S13–S14

Akamai

Best when the enterprise perimeter already uses Akamai. EdgeWorkers can validate JWTs, apply bot signals and sign context.

CloudFront

AWS-native path with WAF, Shield, origin controls and viewer geography headers. Keep personalized AI responses uncacheable.

Varnish

Flexible and capable when already standardized. Requires explicit ownership for patching, policy distribution, observability and availability.

signed-edge-context.json

{
  "requestId": "0190…",
  "country": "CO",
  "region": "ANT",
  "networkRisk": "low",
  "issuedAt": 1784451600,
  "expiresAt": 1784451660,
  "keyId": "edge-context-2026-07"
}

Do not trust client-supplied geography headers

The gateway must strip internal headers, verify the edge signature and fail closed when a jurisdiction is required but cannot be established.

10 · ORGANIZATIONAL BOUNDARIES

Trust scales through account boundaries.

Separate policy ownership, runtime operation, content stewardship, security detection and immutable evidence while keeping the synchronous request path regional and resilient.

Sources S07–S08

Recommended multi-account topology Organizational control without unnecessary synchronous cross-account dependencies

Rendering account boundaries…

Governance owns

Policy source, schemas, release authority, evaluation datasets and signed release manifests.

Platform owns

Runtime, regional policy stores, adapters, SLOs, incident response and deployment automation.

Content owners own

Authoritative rights metadata, classification, legal restrictions, embargoes and allowed AI transformations.

11 · RETRIEVAL PORTFOLIO

One governance path. Multiple retrieval engines.

RAG should be treated as a portfolio of retrieval strategies. The gateway selects lexical, vector, hybrid, graph or lineage traversal according to the question, rights metadata, latency target and evidence requirements.

Sources S19–S22

Governed retrieval router Policy resolves which retrieval paths may execute and which evidence may reach the model

Rendering retrieval portfolio…

Path	Best fit	Governance notes	Exposure posture
OpenSearch Serverless	General enterprise RAG, hybrid retrieval, higher query rates, familiar search operations	Strong metadata design. Authorize returned resources independently. Preferred when Confluence, SharePoint or Salesforce are direct Bedrock data sources.	Internal-ready
Aurora PostgreSQL + pgvector	Relational joins, transactional metadata and vector retrieval in one operational model	Useful where rights and content metadata already live relationally. Avoid coupling all content authorization to database row design.	Internal-ready
Amazon S3 Vectors	Large, cost-sensitive corpora and managed Bedrock ingestion	Current Bedrock integration limits custom metadata per vector. Keep authoritative rights outside the vector index and post-authorize results.	Internal-ready
Neptune Analytics GraphRAG	Entity relationships, multi-hop questions and cross-document context	The automatically generated graph is semantic evidence, not authoritative operational lineage. Pin regions, quotas and ingestion limits during design.	Controlled pilot
Customer-managed retrieval	Specialized ranking, custom parsers, non-Bedrock stores, strict performance or portability needs	Maximum flexibility. Also maximum responsibility for ingestion integrity, filtering, tenancy, deletion, evaluation and evidence.	Controlled pilot

Retrieval evidence

Record query rewrite, selected path, filters, candidate IDs, scores, reranker version, authorization decisions and final citations.

Evaluation by question class

Compare retrieve-only and retrieve-and-generate performance for direct fact, multi-hop, temporal, rights-sensitive and lineage-impact questions.

Portability contract

Expose one provider-neutral retrieval interface so vector-store or graph migrations do not change application or protocol contracts.

Do not choose GraphRAG for every question

Graph expansion increases cost and latency and can amplify incorrect inferred relationships. Use intent classification and measured evaluation to select it only when relationships materially improve the answer.

12 · LINEAGE-AWARE TRIAGE

Use lineage as evidence, not prose.

Amazon DataZone and SageMaker Catalog can capture OpenLineage-compatible job, run and dataset relationships. The governed pathway converts those records into authorized, time-aware evidence for incident triage and change-impact analysis.

Sources S20, S23–S27

Authoritative graph

Operational lineage

OpenLineage events, DataZone lineage nodes and history, Glue and Redshift transformations, ML artifacts and explicit source identifiers.

Trust: factual system-of-record evidence, subject to collection completeness and event quality.

Inferred graph

Semantic GraphRAG

Entities and relationships inferred from documents to improve multi-hop retrieval and cross-document context.

Trust: retrieval aid. Never overwrite authoritative lineage or become the sole basis for a production action.

Lineage triage pathway Read-only evidence first; changes remain separately authorized

Rendering lineage pathway…

Governed lineage tool	Returns	Initial mode
`lineage.resolve_asset`	Canonical asset IDs, catalog references and confidence	Read-only
`lineage.trace_upstream`	Sources, jobs, runs and transformation path to a bounded depth	Read-only
`lineage.trace_downstream`	Consumers, reports, models and likely blast radius	Read-only
`lineage.compare_at_time`	Historical graph differences across an incident or deployment window	Read-only
`lineage.get_quality_signals`	Completeness, timeliness, accuracy and rule failures	Read-only
`lineage.simulate_change_impact`	Proposed blast radius without modifying source systems	Pilot
`lineage.remediate`	Bounded operational change or ticket creation	Separate approval

Recommended storage pattern

Keep DataZone or SageMaker Catalog as the authoritative lineage record. Optionally project normalized nodes and relationships into Neptune Analytics for low-latency traversal, GraphRAG enrichment and custom algorithms. Preserve source identifiers, event timestamps and evidence links.

13 · PROTOCOL FABRIC

Protocols describe interaction. Policy grants authority.

MCP, A2A, AG-UI, HTTP and OpenLineage solve different interoperability problems. None should bypass the gateway’s identity, rights, risk and evidence controls.

Sources S28–S33

Governed protocol fabric Protocol adapters terminate at a common authorization and evidence contract

Rendering protocol fabric…

Protocol or practice	Primary job	Governance position	Recommended use
MCP	Connect model clients and agents to tools, resources and prompts	OAuth 2.1 transport authorization plus gateway action/resource authorization	Preferred developer, assistant and tool interoperability surface
A2A	Agent discovery, delegation, tasks, artifacts and streaming between opaque agents	Authenticate each agent and user delegation. Re-authorize every downstream action.	Internal multi-agent pilots before partner or public federation
AG-UI	Bidirectional events between agents and user-facing applications	Use for status, evidence, consent and human approval—not to move secrets into the UI	Internal agent consoles and review workflows
HTTP / OpenAPI / Smithy	Deterministic service contracts and existing APIs	Remain canonical for many business services; expose selected operations through MCP	Stable backend and write-action contracts
OpenLineage	Describe dataset, job and run lineage events	Ingestion standard for lineage evidence; it is not an agent protocol	Glue, Spark, dbt, Airflow and cross-platform lineage collection
AEO	Answer Engine Optimization: make public content understandable and citable by AI answer systems	Not a transport protocol. Never expose protected content merely to improve discoverability.	Public metadata, canonical sources, structured facts and citation readiness

AEO is not an authorization or agent protocol

Treat it as a public-content publishing discipline. Public abstracts, product metadata and canonical citations may be optimized for answer engines; premium bodies remain behind identity-bound retrieval tools.

14 · CYBER RISK

Exposure changes the risk class.

A read-only internal MCP tool and a public autonomous agent may call the same backend, but they do not have the same threat model. Risk tier determines controls, approvals, testing and allowed protocols.

Sources S10–S11, S28, S31–S32, S39, S42

T0 · Public facts

Anonymous, read-only, curated and non-sensitive. Rate-limited and abuse monitored.

T1 · Internal read

Enterprise identity. Read-only knowledge and lineage. Rights-filtered and fully audited.

T2 · Internal bounded action

Reversible writes, narrow scopes, user confirmation and transaction evidence.

T3 · Privileged or autonomous

Production changes, code execution, sensitive data or agent delegation. Step-up and human approval.

T4 · Public or partner agentic

Internet exposure, subscriptions, third-party hosts or external agents. Full product and security review.

Pathway	Current recommendation	Required before expansion
Internal knowledge retrieval	Ready with controls	Enterprise SSO, rights filters, post-authorization, guardrails, evaluation and immutable evidence
Internal lineage triage	Ready read-only	Scoped asset authorization, source completeness indicators, evidence links and no automated remediation
Developer MCP via CLI/IDE	Ready read-only	Remote authenticated server, private registry, no shared tokens, tool allowlists and local-server governance
Internal A2A delegation	Controlled pilot	Agent identity, delegation chain, task limits, loop controls, downstream authorization and distributed tracing
Write tools and remediation agents	Controlled pilot	Transaction boundaries, dry-run, confirmation, rollback, separation of duties and step-up authentication
Code execution or browser automation	Sandbox only	Ephemeral isolation, network policy, package controls, output scanning, quotas and no ambient credentials
Public remote MCP / ChatGPT App	More vetting	OAuth 2.1, client authentication, penetration test, abuse defense, privacy terms, support model and data deletion
External A2A federation	More vetting	Partner identity, trust agreements, agent-card validation, data contracts, non-repudiation and incident coordination
Paywalled content through assistants	More vetting	Subscription linking, content-license review, anti-scraping, excerpt policy, revocation, checkout and customer support

Risk dimensions

Exposure, data classification, identity assurance, write authority, autonomy, code execution, payment and irreversibility.

Protocol does not set risk

MCP can be safe or dangerous depending on the tool. A2A can be read-only. Classify the effective capability, not the acronym.

Promotion gates

Threat model, adversarial evaluation, privacy review, pentest, operational readiness, rollback and accountable service ownership.

15 · DEVELOPER EXPERIENCE

Bring the governed pathway into the developer loop.

Developers should reach approved knowledge, lineage and operational tools from CLI and IDE clients without copying secrets or configuring provider-specific endpoints.

Sources S29–S30, S34–S35, S41

CLI and IDE identity path One user identity, multiple compatible clients, centrally governed tools

Rendering developer pathway…

Remote first

Use authenticated Streamable HTTP for enterprise tools. Reserve local stdio servers for low-risk development utilities with approved packages and configuration.

Private discovery

Publish approved MCP servers, tools, agents and skills into registries separated by development, QA and production. Treat Agent Registry as preview until promoted.

User attribution

Propagate the human principal to the gateway. Use scoped outbound credentials for downstream systems instead of one broad service identity.

developer-experience.txt

# Product-owned wrapper; provider and protocol details stay behind it
ai-platform login
ai-platform registry search "lineage impact analysis"
ai-platform tools list --environment production
ai-platform invoke lineage.trace_downstream \\
  --asset analytics.customer_360 \\
  --max-depth 4 \\
  --purpose incident-triage

# IDE clients can use the same remote MCP endpoint and OAuth session.

Do not distribute static bearer tokens in IDE configuration

Use OAuth discovery, short-lived access, refresh and revocation. A developer’s tool call must remain attributable to that developer even when the downstream AWS action uses a workload role.

16 · EXTERNAL CONNECTIONS

Let assistants connect. Keep entitlement authoritative.

A ChatGPT App or another assistant host can become a client of the governed pathway. The platform still authenticates the user, verifies subscription and authorizes each content resource before returning it.

Sources S31, S36–S40

Paywalled assistant connection Public discovery, OAuth account linking and merchant-owned checkout

Rendering external connection…

Public discovery tool

Return title, abstract, author, publication date, canonical URL, availability and subscription tier. Never return the protected body.

Entitled retrieval tool

Require OAuth, validate issuer, audience, expiry and scopes, then resolve current subscription and content rights at request time.

Merchant-owned checkout

Use the existing website or application for pricing, tax, payment, refunds and subscription state. Return to the assistant after entitlement changes.

Concern	Required design
Assistant identity	Authenticate the MCP client separately from the end user where supported. Do not treat the host name in a header as trusted.
User identity	OAuth 2.1 authorization code with PKCE, exact audience, scopes, revocation and short-lived tokens.
Subscription	Check the authoritative billing/entitlement service on each protected request or through a tightly bounded cache.
Licensing	Encode allowed summarization, excerpt, transformation, geography, time and machine-use rights per asset.
Anti-scraping	Per-user quotas, anomaly detection, excerpt limits, watermarking or canary markers where legally appropriate, and revocation.
Privacy	Minimize data sent to the host, document retention, support deletion and prevent protected content from entering default logs.
AEO	Optimize public metadata and canonical source pages. Keep premium retrieval behind the same rights-aware API used by internal channels.

The assistant must never become the paywall

The merchant’s identity, billing and rights systems remain authoritative. The assistant is an authenticated client that receives only the content allowed for the current principal, location, purpose and time.

17 · FIRST USE CASE

Start with governed knowledge.

Knowledge answering exercises the full platform boundary: identity, subscription, geography, resource rights, retrieval safety, model routing, citations and evidence.

Sources S03–S06

Capability	Recommended first implementation	Why
Perimeter	Akamai or CloudFront + AWS WAF	Existing enterprise edge or AWS-native control
API	API Gateway REST API	Managed ingress, authorizers, quotas and response streaming path
Runtime	TypeScript Lambda	Fast delivery and AWS SDK integration
Authorization	Amazon Verified Permissions	Externalized Cedar policy
Configuration	AWS AppConfig	Validated, staged operational policy and rollback
Knowledge	S3 + Bedrock Knowledge Bases	Managed retrieval with metadata filters
Rights	DynamoDB or dedicated rights service	Authoritative resource metadata and versioning
Safety	Bedrock Guardrails + application validators	Independent input/output checks and deterministic schema controls
Evidence	Firehose → S3 Object Lock	Durable and tamper-resistant governance trail

Authenticate and derive entitlements

Resolve tenant, durable subject, premium tier, groups and session assurance.

Authorize KnowledgeAnswer

Include purpose, application, subscription, geography and requested collection scope.

Sanitize the question

Block or transform unsafe, secret-bearing or malformed inputs before retrieval.

Retrieve through rights filters

Constrain by tenant, entitlement, validity window, country and allowed purpose.

Re-authorize every candidate

Exclude any chunk whose authoritative rights record does not independently allow access.

Generate and validate

Invoke the approved route, apply output guardrails and validate citations against the authorized source set.

18 · IMAGE EXTENSION

Generated media stays quarantined until cleared.

Image generation adds person, brand, intellectual-property, publication and provenance controls. It should reuse the same action, policy and audit model.

Sources S03, S12

Governed image-generation flow Prompt policy, model adapter, moderation, provenance and release

Rendering image pathway…

Rights

Commercial use, publication, person likeness, protected brands, characters and permitted transformations.

Safety

Prompt controls, image moderation, minor safety, prohibited content and human review thresholds.

Provenance

Provider/model alias, prompt digest, watermark or C2PA status, output digest, retention and release decision.

Route image generation through an alias

Model availability, capability and regional support change. Keep provider identifiers behind the approved model catalog and evaluation process.

19 · RUNTIME

Choose operations proportionate to the workload.

Lambda is the preferred first runtime. ECS or EKS becomes appropriate when sustained connections, long workflows, specialized networking or self-hosted inference justify the operational cost.

Sources S15–S16

Runtime	Use when	Trade-off
Lambda	Initial gateway, bursty traffic, straightforward RAG, policy orchestration	Concurrency, long workflows, connection behavior and cold-path tuning
ECS/Fargate	Sustained traffic, connection pooling, long-lived streams, predictable capacity	More runtime operations and capacity management
EKS/Kubernetes	Existing platform mandate, service mesh, GPU/self-hosted inference, specialized sidecars	Highest security and operational burden

TypeScript first

Best delivery velocity for Lambda, JSON contracts, AWS SDKs, streaming web APIs and shared application types.

Go selectively

Use for profiled high-throughput processors, sustained services, ingestion workers or resource-constrained components.

Do not duplicate the gateway in two languages during phase one

Stabilize the policy contract, adapter boundary and audit schema before creating a second runtime implementation.

20 · ENGINEERING HANDOFF

Make the trust boundary explicit in code.

The initial repository should keep transport, domain policy, provider adapters, retrieval, safety and evidence isolated behind testable interfaces.

Sources S01–S05, S15–S16

Data-plane API

POST /v1/responses
POST /v1/retrieve
POST /v1/images/generations
GET  /v1/image-jobs/{jobId}
GET  /v1/models
GET  /v1/usage

Administrative boundary

Keep policy, model approval, guardrail, rights administration and evaluation APIs separate from the public AI data plane.

Require stronger identity assurance and explicit separation of duties.

gateway-service-layout.txt

src/
├── transport/
│   ├── api-gateway.ts
│   └── response-stream.ts
├── identity/
│   ├── jwt-validator.ts
│   └── principal-normalizer.ts
├── authorization/
│   ├── policy-client.ts
│   ├── action-catalog.ts
│   └── rights-authorizer.ts
├── safety/
│   ├── input-pipeline.ts
│   ├── output-pipeline.ts
│   └── guardrail-client.ts
├── retrieval/
│   ├── metadata-filter.ts
│   ├── knowledge-base.ts
│   └── citation-validator.ts
├── models/
│   ├── router.ts
│   ├── provider-adapter.ts
│   └── providers/bedrock.ts
├── audit/
│   ├── event-schema.ts
│   └── publisher.ts
└── application/
    └── governed-response.ts

Non-negotiable decisions

All AI access crosses the gateway.

Direct provider invocation by product applications is denied by IAM and organizational controls.

Cedar secures application actions and business rights.

IAM secures workloads and AWS resources. Neither replaces the other.

Every retrieved resource is independently authorized.

Vector filters improve precision. They do not replace authorization.

Policies, prompts, routes and guardrails are versioned.

No mutable “latest” dependency in a production decision trail.

Unknown critical context fails closed.

Identity, jurisdiction, content rights and required safety state must be established.

Raw content is excluded from the standard audit path.

Use IDs, digests, policy evidence and tightly governed diagnostic sampling.

Every consequential agent action receives a new decision.

Permission to answer does not imply permission to publish, send, modify or transact.

Image output remains quarantined until cleared.

Safety, rights, provenance and release policy apply before delivery.

21 · ROADMAP

Deliver trust before breadth.

The first release should prove the complete decision and evidence path with one use case. Expansion becomes safer after the core contracts stabilize.

Engineering synthesis

Governed knowledge

Prove the full platform boundary.

Canonical API
OIDC principal
Cedar schema
Rights-aware RAG
Guardrails
Immutable evidence

Lineage and developer access

Add read-only operational context.

OpenLineage ingestion
Lineage triage tools
Private MCP
CLI and IDE access
Registry workflow
RAG evaluation matrix

Platform hardening

Increase resilience and operability.

Multi-region runtime
Regional policy stores
Cost quotas
Provider failover
Advanced evaluations
Formal SLOs

Agents and generated media

Increase capability under bounded risk.

Internal A2A
Write approvals
Sandbox execution
Image rights
Quarantine
Action-level audit

External and paywalled access

Promote only after public review.

Remote MCP app
OAuth linking
Subscription entitlement
External checkout
Abuse defense
Partner operations

Identity issues the ticket. Policy controls the gate. Rights determine the destination. Guardrails inspect the cargo. Evidence records the journey. Operating model

22 · PROVENANCE

Research and source register.

The architecture prioritizes official AWS documentation and recognized security or governance standards. Product availability and regional support must be revalidated during implementation.

Generated 2026-07-19

Method

Synthesized service capabilities, security boundaries, policy design, evaluation practices and deployment trade-offs into an engineering reference architecture.

Evidence preference

Official AWS service documentation first. NIST and OWASP for risk and threat taxonomy. Edge-vendor documentation for edge-specific capabilities.

Freshness rule

Reconfirm model availability, regions, quotas, pricing, API limits and service-specific feature support before production approval.

ID	Source	Supports
S01	Amazon Verified Permissions overview	Externalized fine-grained authorization using Cedar
S02	Verified Permissions policy store schema	Production schema validation and policy consistency
S03	Bedrock ApplyGuardrail API	Independent input and output policy enforcement
S04	Bedrock Knowledge Base retrieval	Retrieval behavior and guardrail boundary
S05	Bedrock RetrievalFilter API	Metadata filtering before candidate return
S06	Amazon Bedrock Knowledge Bases	Managed RAG and knowledge architecture
S07	Amazon S3 Object Lock	WORM protection for governance evidence
S08	CloudTrail log file integrity validation	Digitally signed log digests and integrity checks
S09	Amazon Bedrock evaluations	Model, knowledge base and external RAG evaluation
S10	NIST AI Risk Management Framework	AI risk governance lifecycle
S11	OWASP Top 10 for LLM Applications	Threat model and prompt-injection test coverage
S12	Amazon Titan Image Generator	Image generation, watermarking and C2PA metadata
S13	Akamai EdgeWorkers JWT	Edge JWT verification capability
S14	CloudFront geographic restrictions	Geographic enforcement characteristics and limits
S15	Building Lambda functions with TypeScript	TypeScript Lambda implementation
S16	Lambda response streaming	Streaming through API Gateway proxy integration
S17	AWS AppConfig automatic rollback	CloudWatch-alarm-based configuration rollback
S18	S3 CloudTrail data event logging	Explicit data-plane audit configuration
S19	Create a Bedrock Knowledge Base	Managed vector-store choices, metadata limits and data-source constraints
S20	Bedrock Knowledge Bases GraphRAG	Neptune Analytics semantic graph generation and multi-hop retrieval
S21	GraphRAG metadata filtering best practices	Filter design and GraphRAG query-latency considerations
S22	Bedrock RAG evaluations	Retrieve-only and retrieve-and-generate comparison across RAG sources
S23	Data lineage in Amazon DataZone	OpenLineage compatibility, automatic capture and historical lineage versions
S24	Amazon DataZone lineage APIs	Programmatic node, upstream, downstream and historical lineage access
S25	AWS Glue Data Quality	Catalog-level quality rules and ongoing quality evaluation
S26	AWS Lake Formation	Fine-grained data-lake access control distinct from lineage capture
S27	SageMaker ML Lineage Tracking	Dataset, training, model and deployment lineage for ML workflows
S28	Amazon Bedrock AgentCore Gateway	Managed ingress and egress gateway for tools, agents and models
S29	AgentCore Runtime protocol support	HTTP, MCP, A2A and bidirectional agent runtime patterns
S30	AWS Agent Registry	Preview registry for curated MCP servers, agents, skills and custom resources
S31	MCP authorization specification	OAuth 2.1 resource-server and client authorization requirements
S32	Agent2Agent protocol specification	Agent discovery, tasks, artifacts, streaming and security metadata
S33	AG-UI protocol	Event-based agent-to-user-interface communication
S34	Using MCP with Amazon Q Developer	Local and remote MCP access from CLI and IDE
S35	AgentCore MCP server for coding assistants	Kiro, Cursor, Claude Code and Amazon Q CLI development support
S36	Connect an MCP app to ChatGPT	Remote HTTPS MCP app connection and developer-mode testing
S37	OpenAI Apps SDK authentication	OAuth authorization code, PKCE and per-request token verification
S38	OpenAI Apps SDK monetization	Merchant-hosted external checkout as the recommended monetization path
S39	OpenAI Apps SDK security and privacy	Least privilege, consent, prompt injection, logging and launch review
S40	MCP Apps compatibility in ChatGPT	Host-neutral MCP Apps UI fields with optional ChatGPT extensions
S41	AgentCore Gateway inbound authorization	JWT and MCP-compliant authentication discovery for gateway clients
S42	AgentCore Code Interpreter	Isolated agent code execution, network modes and CloudTrail observability

Claim traceability

Each major section declares source IDs in its header. The source registry maps those IDs to authoritative documentation and the specific architectural claim they support.

Trust is the platform boundary.

Authentication

Authorization

Content rights

AI safety

Fail closed

No direct provider access

Data plane

Control plane

Evidence plane

Before retrieval

Before inference

Before release

Client-facing aliases

Provider adapter ownership

Verified Permissions

AWS AppConfig

Guardrail registry

Operational telemetry

Governance evidence

Authorization evaluations

Safety evaluations

Quality evaluations

Akamai

CloudFront

Varnish

Governance owns

Platform owns

Content owners own

Retrieval evidence

Evaluation by question class

Portability contract

Operational lineage

Semantic GraphRAG

Risk dimensions

Protocol does not set risk

Promotion gates

Remote first

Private discovery

User attribution

Public discovery tool

Entitled retrieval tool

Merchant-owned checkout

Rights

Safety

Provenance

TypeScript first

Go selectively

Data-plane API

Administrative boundary

Non-negotiable decisions

Governed knowledge

Lineage and developer access

Platform hardening

Agents and generated media

External and paywalled access

Method

Evidence preference

Freshness rule