Architecture Overview

Understanding Agent Kernel's architecture helps you build robust, scalable AI agent systems.

High-Level Architecture

Agent Kernel sits between your agent logic (written in any supported framework) and the surfaces that expose it to the world: CLI, REST/SSE, WebSocket, MCP, A2A, messaging platforms, and cloud deployment targets.

Layers at a glance:

Layer	Components	Responsibility
Application	Your agents and tools	Domain logic, written with any supported framework
Core	Module, Agent, Runner, Session, Runtime, hooks, AgentService/ChatService	Framework-agnostic orchestration, state, and the run/stream pipeline
Framework adapters	OpenAI Agents SDK, CrewAI, LangGraph, Google ADK, Smolagents	Wrap native agents behind the core abstractions
System plugins	Guardrails, multimodal, conversation threads, knowledge bases, tracing	Cross-cutting features implemented as hooks, tools, and services
State stores	In-memory, Redis, Valkey, DynamoDB, Cosmos DB, Firestore	Pluggable persistence for sessions, threads, attachments, and responses
Execution surfaces	CLI, REST (JSON + SSE), WebSocket, MCP, A2A, messaging integrations, cloud deployments	How requests reach the runtime and how replies get back out

Key Design Principles

1. Framework Agnostic

All core abstractions (Session, Agent, Runner, Module, Runtime) are framework-independent. Framework-specific logic lives exclusively in adapter modules; the same hooks, tools, session stores, and deployment targets work with every supported framework, and agents from different frameworks can run side by side in one runtime.

2. Minimal Overhead

The kernel adds minimal latency and complexity; it's primarily orchestration and state management around your framework's native execution.

3. Config-Driven Behavior

All runtime behavior is governed by AKConfig (Pydantic-based), loaded from YAML/JSON files and environment variables (AK_ prefix, __ for nesting). The same application code switches between synchronous REST, SSE streaming, queue-backed async, and WebSocket delivery purely through configuration.

4. Production Ready

Built-in support for:

• Multi-cloud session persistence (AWS, Azure, GCP)
• Token-level streaming (SSE over REST, WebSocket on AWS serverless)
• Queue-based scalable execution (SQS-backed, on Lambda and ECS)
• Input/output guardrails and PII redaction
• Multi-agent coordination and multimodal attachments
• Observability and tracing (Langfuse, OpenLLMetry, Logfire)

5. Extensible

Pluggable via well-defined interfaces:

• New framework adapters (Agent/Runner/Module/ToolBuilder subclasses)
• Custom session, thread, and attachment storage backends
• Custom guardrail and tracing providers
• Pre/post execution hooks
• Knowledge base backends

The Run Pipeline

Runtime.run() is the heart of every execution surface. It wraps the framework call with session locking, hooks, and persistence:

Key properties:

• Session locking: async with session serializes concurrent requests for the same session and sets the session as the current context (Session.current() works anywhere inside the run).
• Pre-hooks can rewrite the request list or halt execution by returning an AgentReply (this is how input guardrails block a request before the LLM sees it).
• Post-hooks can transform the reply (output guardrails, disclaimers, redaction).
• Persistence and cleanup always run: the session is stored and the volatile cache cleared even if the run raises.

The Streaming Pipeline

Runtime.stream() is the streaming counterpart, sharing the same pre-hook pipeline but yielding StreamChunk objects as tokens arrive:

• Each StreamChunk carries delta, done, error, and session_id fields.
• Post-hooks can filter or redact individual tokens via on_stream_chunk(); returning None drops the token.
• Delivery depends on the surface: the REST API serves chunks as Server-Sent Events (text/event-stream); AWS serverless WebSocket mode pushes each chunk as a separate STREAM_CHUNK WebSocket message (optionally through SQS queues).
• OpenAI Agents SDK, LangGraph, and Google ADK stream natively; CrewAI and Smolagents do not support token streaming (their runners raise NotImplementedError in stream mode).

See Execution Flow for the full request lifecycle including the queue-based and WebSocket paths.

Scalable Execution Topologies

Beyond the in-process pipeline above, Agent Kernel ships deployment adapters that decouple request ingestion from agent execution using SQS queues. The same execution config block drives both:

• On AWS Lambda, the three roles are three Lambda functions wired by SQS event source mappings; see AWS Serverless.
• On AWS ECS Fargate, the request handler and response handler run as threads inside one IO container, and the agent runner is a separate auto-scaling ECS service with a pool of consumer threads; see AWS Containerized and the Queue Mode Guide.
• The client receives the reply either by polling the response store (rest_sync waits server-side, rest_async polls with a request_id) or, on AWS serverless, by WebSocket push (async for full responses, stream for token-level chunks). ECS queue mode delivers replies via the response store only.

Next Steps

• Execution Flow: request lifecycle across all execution modes
• Session Management: detailed session configuration
• Memory Management: advanced memory features
• Knowledge Bases: knowledge backends and KB routing
• Deployment Overview: choosing a deployment mode