Governed AI Gateway · AWS Reference Architecture

01 · FOUNDATION

Four decisions. Four owners.

The platform remains governable when authentication, authorization, content rights and AI safety are modeled as separate decisions with independent evidence.

Sources S01–S03

Authentication

Question: Who is making the request?

Enterprise OIDC. Durable subject ID. Tenant. Session assurance. Authentication time.

Authorization

Question: May this principal perform this platform action?

Amazon Verified Permissions. Cedar policies. Default-deny. Explicit action and resource.

Content rights

Question: May this principal use this asset now, here and for this purpose?

Entitlement, subscription, time window, country, classification and derivative-use terms.

AI safety

Question: Is this input, retrieved context, action or output acceptable?

Guardrails, schema validation, sanitizers, prompt-injection defenses and evaluations.

Models generate content. They never authorize access. Primary security invariant

Fail closed

Deny when identity, tenant, rights, policy version, geography or required safety enforcement cannot be established.

No direct provider access

Applications do not receive broad model invocation permissions. The governed gateway is the approved invocation boundary.

02 · SYSTEM TOPOLOGY

Separate the path from the policy.

The data plane handles low-latency requests. The control plane governs policies, models, prompts, guardrails and evaluations without coupling changes to application code.

Sources S01–S06

Governed AI Gateway topology Edge, data plane, control plane, knowledge, models and evidence

Rendering architecture…

Data plane

Identity, request normalization, authorization, retrieval, provider invocation, output enforcement and evidence.

Control plane

Policy source, schemas, configuration, model catalog, prompts, guardrail versions, evaluations and approvals.

Evidence plane

Operational telemetry and immutable governance events, separated from raw prompt or response retention.

Lambda-first, not Lambda-only

The contract, policy model, audit schema and provider adapters must remain portable to ECS, EKS or another Kubernetes runtime.

03 · TRUST PATH

Authorize before context becomes capability.

Every request follows the same order. Each stage emits enough evidence to explain why the final response was allowed, constrained, blocked or denied.

Sources S03–S05

Request sequence Knowledge answer with independent input, retrieval and output controls

Rendering request path…

Validate edge context

Strip spoofable headers. Verify signature, key ID, issue time, expiry and request binding.

Normalize identity

Convert provider claims into a stable platform principal. Email is not the durable key.

Authorize the action

Evaluate principal, tenant, purpose, subscription, country and application against Cedar policy.

Inspect input

Enforce schema, size, Unicode, secret, PII, content and prompt-injection controls before retrieval.

Authorize retrieved resources

Metadata narrows candidates. Per-resource authorization remains the final boundary.

Invoke through an adapter

Resolve a capability alias to an approved provider, model, region, quota and guardrail version.

Validate output

Recheck safety, structured output, URLs, citations, data leakage and rights-dependent excerpt limits.

Publish evidence

Record decision provenance, content IDs, policy hashes, model route and usage without default raw-content logging.

04 · RIGHTS ENGINE

Entitlement is data, not code.

Premium access, person-specific grants, embargoes, time windows, purpose limits and geographic restrictions should be editable as rights data and policy.

Sources S01, S02, S05

Rights-aware retrieval Retrieval filters improve precision; resource authorization enforces security

Rendering rights model…

Rights dimension	Example attribute	Enforcement
Identity	`allowedPrincipalIds`, groups, tenant	Per-resource Cedar decision
Subscription	`minimumSubscription = premium`	Policy plus entitlement record
Time	`validFrom`, `validUntil`, embargo	Signed request time in context
Geography	Allowed and denied countries or legal regions	Signed edge context, rechecked at origin
Purpose	Support, research, summary, publication	Explicit action and request purpose
AI usage	Retrieval, summarization, transformation, training	Separate rights flags and actions
Output	Maximum excerpt length, citation requirement	Prompt and response post-processing

content-rights.ts

export interface ContentRights {
  contentId: string;
  contentVersion: string;
  tenantId: string;
  classification: "public" | "internal" | "confidential" | "restricted";

  requiredEntitlement?: string;
  minimumSubscription: "free" | "standard" | "premium" | "enterprise";
  allowedPrincipalIds?: string[];
  allowedGroups?: string[];

  allowedCountries?: string[];
  deniedCountries?: string[];
  validFromEpoch?: number;
  validUntilEpoch?: number;
  allowedPurposes: string[];

  aiRetrievalAllowed: boolean;
  summarizationAllowed: boolean;
  derivativeGenerationAllowed: boolean;
  modelTrainingAllowed: boolean;

  maximumExcerptCharacters?: number;
  policyVersion: string;
}

Retrieval filters are not the final authorization boundary

Use filters to reduce the candidate set. Load authoritative metadata and authorize each returned content resource before it enters a prompt.

05 · SAFETY

Guard the whole context path.

Input guardrails alone are insufficient. Retrieved documents, structured tool arguments, generated URLs, image prompts and final output each require separate enforcement.

Sources S03, S04, S10, S11

Before retrieval

Schema and payload limits
Unicode normalization
Secret and PII detection
Direct prompt-injection analysis
Denied topics and abuse controls

Before inference

Authorized context only
Instruction/data separation
Active markup removal
Indirect prompt-injection analysis
Protected prompt template

Before release

Output guardrail
Structured schema validation
URL and HTML sanitization
Citation authorization check
Rights-based excerpt enforcement

Layer	Primary controls	Failure posture
Edge	WAF, bot controls, coarse quotas, size limits	Reject obvious abuse early
Identity	JWT claims, issuer, audience, session assurance	Deny unverified identity
Request	JSON schema, normalization, replay and idempotency	Reject malformed request
Retrieval	Ingestion quarantine, metadata filters, resource authorization	Exclude unauthorized context
Model	Approved alias, token limits, region policy, timeout	No uncontrolled fallback
Output	Guardrail, data loss prevention, citation and format checks	Block, transform or safe refusal
Evidence	Redaction, digests, immutable audit archive	Do not log raw sensitive content by default

Retrieved content is untrusted input

A knowledge document can contain instructions designed to override the system, exfiltrate data or trigger tools. It must never inherit authority from its storage location.

06 · MODEL PORTABILITY

Route capabilities, not model IDs.

Clients request governed capabilities. The platform resolves provider, model, region, quota, guardrail and cost profile at runtime.

Sources S03, S12

Client-facing aliases

knowledge-balanced, regulated-text, low-latency, approved-image.

Aliases remain stable when providers, versions or regions change.

Provider adapter ownership

Credential handling, request translation, streaming normalization, retries, usage extraction and provider error mapping.

Authorization remains outside the adapter.

model-provider-adapter.ts

export type AICapability =
  | "chat"
  | "structured-output"
  | "embeddings"
  | "image-generation"
  | "tool-use";

export interface ModelProviderAdapter {
  readonly providerId: string;

  supports(capability: AICapability): boolean;

  invoke(
    request: GovernedAIRequest,
    context: GovernedContext,
    signal: AbortSignal
  ): AsyncIterable<GatewayEvent>;
}

No silent safety downgrade

A provider outage may trigger an approved route change. It must not remove a required guardrail, broaden data residency or reduce the authorization posture.

07 · CHANGE GOVERNANCE

Policy changes are production changes.

Cedar, AppConfig, prompts, guardrails and model routes require schema validation, regression tests, provenance and staged release—even when no application code changes.

Sources S01–S03

Governance release pipeline Policy-as-code with evaluation gates, approval and automatic rollback

Rendering control plane…

Verified Permissions

Fine-grained action and resource policy. Use a strict Cedar schema and policy validation for production stores.

AWS AppConfig

Model routes, quotas, country controls, feature flags and kill switches with validators, staged deployment and rollback alarms.

Guardrail registry

Named, versioned safety policies. Routes pin a tested guardrail version rather than resolving an unversioned mutable configuration.

repository-shape.txt

ai-governance/
├── cedar/
│   ├── schema.json
│   ├── platform/
│   ├── knowledge/
│   └── images/
├── appconfig/
│   ├── model-routes.json
│   ├── quotas.json
│   └── emergency-controls.json
├── guardrails/
├── prompts/
├── evals/
│   ├── authorization/
│   ├── safety/
│   ├── retrieval/
│   └── model-quality/
└── provenance/
    └── release-manifest.schema.json

08 · ASSURANCE

Every answer needs an evidence trail.

The system must reconstruct who requested what, which rights and policies applied, what content was authorized, which route ran and what safety decisions occurred.

Sources S07–S09

Operational telemetry

CloudWatch metrics and logs, X-Ray traces, latency, errors, quotas, provider health and cost units.

Governance evidence

Structured events through Data Firehose into a dedicated, KMS-encrypted S3 archive protected by Object Lock.

Evidence category	Record	Avoid by default
Identity	Principal hash, tenant, application, session assurance	Email or unnecessary profile data
Authorization	Decision, policy store, bundle hash, determining policies	Opaque “allowed=true” only
Content	Requested, allowed and denied content IDs; rights versions	Full protected document text
AI route	Provider, alias, resolved model, prompt and guardrail versions	Unversioned “latest” references
Usage	Input/output tokens, latency, cost units, result class	Secrets, access tokens and raw credentials
Integrity	Input/output digests and archive checks	Raw prompt/response retention as the standard path

Authorization evaluations

Zero cross-tenant, premium, person, embargo, time-window or geography leakage.

Safety evaluations

Direct and indirect injection, secret extraction, PII, jailbreaks, malicious URLs and tool misuse.

Quality evaluations

Groundedness, citation precision, retrieval recall, refusal correctness, answer relevance and hallucination rate.

Evaluate policy and retrieval changes—not only models

A safer model cannot compensate for an authorization regression or a metadata-filter bug. Release gates must cover the entire trust path.

09 · EDGE

Enrich at the edge. Decide at the origin.

Edge infrastructure can reduce abuse and attach signed geography or network risk. It should not become the sole authority for legal or rights-sensitive access.

Sources S13–S14

Akamai

Best when the enterprise perimeter already uses Akamai. EdgeWorkers can validate JWTs, apply bot signals and sign context.

CloudFront

AWS-native path with WAF, Shield, origin controls and viewer geography headers. Keep personalized AI responses uncacheable.

Varnish

Flexible and capable when already standardized. Requires explicit ownership for patching, policy distribution, observability and availability.

signed-edge-context.json

{
  "requestId": "0190…",
  "country": "CO",
  "region": "ANT",
  "networkRisk": "low",
  "issuedAt": 1784451600,
  "expiresAt": 1784451660,
  "keyId": "edge-context-2026-07"
}

Do not trust client-supplied geography headers

The gateway must strip internal headers, verify the edge signature and fail closed when a jurisdiction is required but cannot be established.

10 · ORGANIZATIONAL BOUNDARIES

Trust scales through account boundaries.

Separate policy ownership, runtime operation, content stewardship, security detection and immutable evidence while keeping the synchronous request path regional and resilient.

Sources S07–S08

Recommended multi-account topology Organizational control without unnecessary synchronous cross-account dependencies

Rendering account boundaries…

Governance owns

Policy source, schemas, release authority, evaluation datasets and signed release manifests.

Platform owns

Runtime, regional policy stores, adapters, SLOs, incident response and deployment automation.

Content owners own

Authoritative rights metadata, classification, legal restrictions, embargoes and allowed AI transformations.

11 · FIRST USE CASE

Start with governed knowledge.

Knowledge answering exercises the full platform boundary: identity, subscription, geography, resource rights, retrieval safety, model routing, citations and evidence.

Sources S03–S06

Capability	Recommended first implementation	Why
Perimeter	Akamai or CloudFront + AWS WAF	Existing enterprise edge or AWS-native control
API	API Gateway REST API	Managed ingress, authorizers, quotas and response streaming path
Runtime	TypeScript Lambda	Fast delivery and AWS SDK integration
Authorization	Amazon Verified Permissions	Externalized Cedar policy
Configuration	AWS AppConfig	Validated, staged operational policy and rollback
Knowledge	S3 + Bedrock Knowledge Bases	Managed retrieval with metadata filters
Rights	DynamoDB or dedicated rights service	Authoritative resource metadata and versioning
Safety	Bedrock Guardrails + application validators	Independent input/output checks and deterministic schema controls
Evidence	Firehose → S3 Object Lock	Durable and tamper-resistant governance trail

Authenticate and derive entitlements

Resolve tenant, durable subject, premium tier, groups and session assurance.

Authorize KnowledgeAnswer

Include purpose, application, subscription, geography and requested collection scope.

Sanitize the question

Block or transform unsafe, secret-bearing or malformed inputs before retrieval.

Retrieve through rights filters

Constrain by tenant, entitlement, validity window, country and allowed purpose.

Re-authorize every candidate

Exclude any chunk whose authoritative rights record does not independently allow access.

Generate and validate

Invoke the approved route, apply output guardrails and validate citations against the authorized source set.

12 · IMAGE EXTENSION

Generated media stays quarantined until cleared.

Image generation adds person, brand, intellectual-property, publication and provenance controls. It should reuse the same action, policy and audit model.

Sources S03, S12

Governed image-generation flow Prompt policy, model adapter, moderation, provenance and release

Rendering image pathway…

Rights

Commercial use, publication, person likeness, protected brands, characters and permitted transformations.

Safety

Prompt controls, image moderation, minor safety, prohibited content and human review thresholds.

Provenance

Provider/model alias, prompt digest, watermark or C2PA status, output digest, retention and release decision.

Route image generation through an alias

Model availability, capability and regional support change. Keep provider identifiers behind the approved model catalog and evaluation process.

13 · RUNTIME

Choose operations proportionate to the workload.

Lambda is the preferred first runtime. ECS or EKS becomes appropriate when sustained connections, long workflows, specialized networking or self-hosted inference justify the operational cost.

Sources S15–S16

Runtime	Use when	Trade-off
Lambda	Initial gateway, bursty traffic, straightforward RAG, policy orchestration	Concurrency, long workflows, connection behavior and cold-path tuning
ECS/Fargate	Sustained traffic, connection pooling, long-lived streams, predictable capacity	More runtime operations and capacity management
EKS/Kubernetes	Existing platform mandate, service mesh, GPU/self-hosted inference, specialized sidecars	Highest security and operational burden

TypeScript first

Best delivery velocity for Lambda, JSON contracts, AWS SDKs, streaming web APIs and shared application types.

Go selectively

Use for profiled high-throughput processors, sustained services, ingestion workers or resource-constrained components.

Do not duplicate the gateway in two languages during phase one

Stabilize the policy contract, adapter boundary and audit schema before creating a second runtime implementation.

14 · ENGINEERING HANDOFF

Make the trust boundary explicit in code.

The initial repository should keep transport, domain policy, provider adapters, retrieval, safety and evidence isolated behind testable interfaces.

Sources S01–S05, S15–S16

Data-plane API

POST /v1/responses
POST /v1/retrieve
POST /v1/images/generations
GET  /v1/image-jobs/{jobId}
GET  /v1/models
GET  /v1/usage

Administrative boundary

Keep policy, model approval, guardrail, rights administration and evaluation APIs separate from the public AI data plane.

Require stronger identity assurance and explicit separation of duties.

gateway-service-layout.txt

src/
├── transport/
│   ├── api-gateway.ts
│   └── response-stream.ts
├── identity/
│   ├── jwt-validator.ts
│   └── principal-normalizer.ts
├── authorization/
│   ├── policy-client.ts
│   ├── action-catalog.ts
│   └── rights-authorizer.ts
├── safety/
│   ├── input-pipeline.ts
│   ├── output-pipeline.ts
│   └── guardrail-client.ts
├── retrieval/
│   ├── metadata-filter.ts
│   ├── knowledge-base.ts
│   └── citation-validator.ts
├── models/
│   ├── router.ts
│   ├── provider-adapter.ts
│   └── providers/bedrock.ts
├── audit/
│   ├── event-schema.ts
│   └── publisher.ts
└── application/
    └── governed-response.ts

Non-negotiable decisions

All AI access crosses the gateway.

Direct provider invocation by product applications is denied by IAM and organizational controls.

Cedar secures application actions and business rights.

IAM secures workloads and AWS resources. Neither replaces the other.

Every retrieved resource is independently authorized.

Vector filters improve precision. They do not replace authorization.

Policies, prompts, routes and guardrails are versioned.

No mutable “latest” dependency in a production decision trail.

Unknown critical context fails closed.

Identity, jurisdiction, content rights and required safety state must be established.

Raw content is excluded from the standard audit path.

Use IDs, digests, policy evidence and tightly governed diagnostic sampling.

Every consequential agent action receives a new decision.

Permission to answer does not imply permission to publish, send, modify or transact.

Image output remains quarantined until cleared.

Safety, rights, provenance and release policy apply before delivery.

15 · ROADMAP

Deliver trust before breadth.

The first release should prove the complete decision and evidence path with one use case. Expansion becomes safer after the core contracts stabilize.

Engineering synthesis

Governed knowledge

Prove the full platform boundary.

Canonical API
OIDC principal
Cedar schema
Rights-aware retrieval
Input/output guardrails
Immutable evidence

Platform hardening

Increase resilience and operability.

Multi-region runtime
Regional policy stores
Cost and usage quotas
Provider failover
Advanced evaluations
Formal SLOs

Image generation

Extend the same governance model.

Image actions and rights
Async job orchestration
Output moderation
Provenance controls
Quarantine and review
Signed delivery

Tools and agents

Authorize each consequential action.

Tool registry
Purpose-bound scopes
Step-up authentication
Approval workflows
Action-level audit
Revocation and kill switches

Identity issues the ticket. Policy controls the gate. Rights determine the destination. Guardrails inspect the cargo. Evidence records the journey. Operating model

16 · PROVENANCE

Research and source register.

The architecture prioritizes official AWS documentation and recognized security or governance standards. Product availability and regional support must be revalidated during implementation.

Generated 2026-07-19

Method

Synthesized service capabilities, security boundaries, policy design, evaluation practices and deployment trade-offs into an engineering reference architecture.

Evidence preference

Official AWS service documentation first. NIST and OWASP for risk and threat taxonomy. Edge-vendor documentation for edge-specific capabilities.

Freshness rule

Reconfirm model availability, regions, quotas, pricing, API limits and service-specific feature support before production approval.

ID	Source	Supports
S01	Amazon Verified Permissions overview	Externalized fine-grained authorization using Cedar
S02	Verified Permissions policy store schema	Production schema validation and policy consistency
S03	Bedrock ApplyGuardrail API	Independent input and output policy enforcement
S04	Bedrock Knowledge Base retrieval	Retrieval behavior and guardrail boundary
S05	Bedrock RetrievalFilter API	Metadata filtering before candidate return
S06	Amazon Bedrock Knowledge Bases	Managed RAG and knowledge architecture
S07	Amazon S3 Object Lock	WORM protection for governance evidence
S08	CloudTrail log file integrity validation	Digitally signed log digests and integrity checks
S09	Amazon Bedrock evaluations	Model, knowledge base and external RAG evaluation
S10	NIST AI Risk Management Framework	AI risk governance lifecycle
S11	OWASP Top 10 for LLM Applications	Threat model and prompt-injection test coverage
S12	Amazon Titan Image Generator	Image generation, watermarking and C2PA metadata
S13	Akamai EdgeWorkers JWT	Edge JWT verification capability
S14	CloudFront geographic restrictions	Geographic enforcement characteristics and limits
S15	Building Lambda functions with TypeScript	TypeScript Lambda implementation
S16	Lambda response streaming	Streaming through API Gateway proxy integration
S17	AWS AppConfig automatic rollback	CloudWatch-alarm-based configuration rollback
S18	S3 CloudTrail data event logging	Explicit data-plane audit configuration

Claim traceability

Each major section declares source IDs in its header. The source registry maps those IDs to authoritative documentation and the specific architectural claim they support.

Trust is the platform boundary.

Authentication

Authorization

Content rights

AI safety

Fail closed

No direct provider access

Data plane

Control plane

Evidence plane

Before retrieval

Before inference

Before release

Client-facing aliases

Provider adapter ownership

Verified Permissions

AWS AppConfig

Guardrail registry

Operational telemetry

Governance evidence

Authorization evaluations

Safety evaluations

Quality evaluations

Akamai

CloudFront

Varnish

Governance owns

Platform owns

Content owners own

Rights

Safety

Provenance

TypeScript first

Go selectively

Data-plane API

Administrative boundary

Non-negotiable decisions

Governed knowledge

Platform hardening

Image generation

Tools and agents

Method

Evidence preference

Freshness rule