Middleware Lifecycle

Complete reference of all middleware hooks, typed contexts, and execution flow.

Execution Flow

User Message
    ↓
BeforeMessageTurnAsync(BeforeMessageTurnContext)
    ↓
[LOOP] BeforeIterationAsync(BeforeIterationContext)
    ↓
WrapModelCallAsync(ModelRequest) OR WrapModelCallStreamingAsync(ModelRequest)
    ↓
LLM Response
    ↓
BeforeToolExecutionAsync(BeforeToolExecutionContext)
    ↓
BeforeParallelBatchAsync(BeforeParallelBatchContext) [if parallel tools]
    ↓
[LOOP] BeforeFunctionAsync(BeforeFunctionContext)
    ↓
WrapFunctionCallAsync(FunctionRequest)
    ↓
Function Result
    ↓
AfterFunctionAsync(AfterFunctionContext)
    ↓
AfterIterationAsync(AfterIterationContext)
    ↓
AfterMessageTurnAsync(AfterMessageTurnContext)
    ↓
Final Response

[ON ERROR] OnErrorAsync(ErrorContext) - runs on ANY error

Execution Order

Before* hooks - Run in registration order
After* hooks - Run in REVERSE order (stack unwinding)
OnErrorAsync - Runs in REVERSE order (error unwinding)
Wrap* hooks - Onion architecture (last registered is outermost)

Onion Architecture (Wrap Hooks)

csharp

.WithMiddleware(new LoggingMiddleware())   // Inner
.WithMiddleware(new CachingMiddleware())   // Middle
.WithMiddleware(new RetryMiddleware())     // Outer

Execution flow:

RetryMiddleware.WrapModelCallAsync()
  → CachingMiddleware.WrapModelCallAsync()
    → LoggingMiddleware.WrapModelCallAsync()
      → Actual LLM call

Turn Level Hooks

BeforeMessageTurnAsync

csharp

Task BeforeMessageTurnAsync(
    BeforeMessageTurnContext context,
    CancellationToken cancellationToken)

When: Before processing user message Context Properties:

UserMessage - User's message (string)
ConversationHistory - Prior messages (mutable list)
RunOptions - User's original options (read-only)
State - Agent state (read via .State, update via .UpdateState())

Use Cases: RAG injection, memory retrieval, context augmentation

Example:

csharp

public class MemoryMiddleware : IAgentMiddleware
{
    public async Task BeforeMessageTurnAsync(
        BeforeMessageTurnContext context,
        CancellationToken ct)
    {
        var memories = await _store.GetRelevant(context.UserMessage, ct);

        context.ConversationHistory.Insert(0, new ChatMessage(
            ChatRole.System,
            $"Relevant memory: {string.Join(", ", memories)}"
        ));
    }
}

AfterMessageTurnAsync

csharp

Task AfterMessageTurnAsync(
    AfterMessageTurnContext context,
    CancellationToken cancellationToken)

When: After turn completes Context Properties:

FinalResponse - Assistant's final message
TurnHistory - All messages from this turn (mutable list)
RunOptions - User's original options (read-only)
State - Agent state

Always runs - Even if operations failed

Use Cases: Memory extraction, analytics, turn logging

Example:

csharp

public Task AfterMessageTurnAsync(AfterMessageTurnContext context, CancellationToken ct)
{
    _logger.LogInformation(
        "Turn completed. User: {User}, Agent: {Agent}",
        context.TurnHistory.First(m => m.Role == ChatRole.User).Text,
        context.FinalResponse.Text
    );

    return Task.CompletedTask;
}

Iteration Level Hooks

BeforeIterationAsync

csharp

Task BeforeIterationAsync(
    BeforeIterationContext context,
    CancellationToken cancellationToken)

When: Before each LLM call Context Properties:

Iteration - Current iteration number (0-based)
Messages - Mutable message list (modify before LLM sees them)
Options - Mutable chat options (modify temperature, etc.)
RunOptions - Read-only user options

Control Flow:

Set SkipLLMCall = true to skip this LLM call
Set OverrideResponse to provide cached response

Use Cases: History reduction, dynamic instructions, prompt optimization

Example:

csharp

public Task BeforeIterationAsync(BeforeIterationContext context, CancellationToken ct)
{
    // Add retry instruction on subsequent iterations
    if (context.Iteration > 0)
    {
        context.Messages.Insert(0, new ChatMessage(
            ChatRole.System,
            "Previous approach failed. Try a different tool."
        ));
    }

    return Task.CompletedTask;
}

WrapModelCallAsync (Simple Pattern - Recommended)

csharp

Task<ModelResponse> WrapModelCallAsync(
    ModelRequest request,
    Func<ModelRequest, Task<ModelResponse>> handler,
    CancellationToken cancellationToken)

When: Wraps LLM call (90% of use cases) Request Properties (Immutable):

Model - Chat model instance
Messages - Read-only message list
Options - Chat options
State - Agent state
Iteration - Current iteration

Immutable Pattern:

csharp

var newRequest = request.Override(
    messages: request.Messages.Append(msg).ToList(),
    options: new ChatOptions { Temperature = 0.5f }
);
return await handler(newRequest);

Use Cases: Retry, caching, request modification, fallback models

Example - Retry:

csharp

public async Task<ModelResponse> WrapModelCallAsync(
    ModelRequest request,
    Func<ModelRequest, Task<ModelResponse>> handler,
    CancellationToken ct)
{
    for (int i = 0; i < 3; i++)
    {
        try
        {
            return await handler(request);
        }
        catch (Exception ex) when (i < 2 && ShouldRetry(ex))
        {
            await Task.Delay(TimeSpan.FromSeconds(Math.Pow(2, i)), ct);
        }
    }

    return await handler(request); // Final attempt
}

private bool ShouldRetry(Exception ex) =>
    ex is HttpRequestException or TaskCanceledException;

WrapModelCallStreamingAsync (Advanced Pattern)

csharp

IAsyncEnumerable<ChatResponseUpdate>? WrapModelCallStreamingAsync(
    ModelRequest request,
    Func<ModelRequest, IAsyncEnumerable<ChatResponseUpdate>> handler,
    [EnumeratorCancellation] CancellationToken cancellationToken)

When: Need streaming control (rare - 10% of cases) Return null to use simple pattern instead

Use Cases: Token counting, mid-stream transformation, synthetic updates, streaming cache

Example - Token Counting:

csharp

public async IAsyncEnumerable<ChatResponseUpdate> WrapModelCallStreamingAsync(
    ModelRequest request,
    Func<ModelRequest, IAsyncEnumerable<ChatResponseUpdate>> handler,
    [EnumeratorCancellation] CancellationToken ct)
{
    int tokens = 0;

    await foreach (var update in handler(request).WithCancellation(ct))
    {
        if (update.Contents != null)
        {
            foreach (var content in update.Contents)
                if (content is TextContent text)
                    tokens += EstimateTokens(text.Text);
        }

        yield return update;
    }

    // Emit total after stream completes
    _telemetry.RecordTokens(tokens);
}

BeforeToolExecutionAsync

csharp

Task BeforeToolExecutionAsync(
    BeforeToolExecutionContext context,
    CancellationToken cancellationToken)

When: After LLM returns, BEFORE tools execute Context Properties:

Response - LLM's response message
ToolCalls - List of tool calls requested
RunOptions - Read-only user options

Control Flow:

Set SkipToolExecution = true to skip ALL tools

Use Cases: Circuit breaker, batch validation, cost estimation

Example:

csharp

public Task BeforeToolExecutionAsync(BeforeToolExecutionContext context, CancellationToken ct)
{
    // Circuit breaker: detect repeated identical calls
    foreach (var call in context.ToolCalls)
    {
        var signature = ComputeSignature(call);
        var count = context.GetMiddlewareState<CircuitBreakerState>()?
            .GetCount(call.Name, signature) ?? 0;

        if (count >= 3)
        {
            context.SkipToolExecution = true;
            context.Emit(new TextDeltaEvent
            {
                Text = $"⛔ Stopping repeated calls to {call.Name}"
            });
            break;
        }
    }

    return Task.CompletedTask;
}

AfterIterationAsync

csharp

Task AfterIterationAsync(
    AfterIterationContext context,
    CancellationToken cancellationToken)

When: After all tools complete Context Properties:

Iteration - Iteration number
ToolResults - Results from tool executions
RunOptions - Read-only user options

Always runs - Even if tools failed

Use Cases: Error tracking, result validation, state updates

Example:

csharp

public Task AfterIterationAsync(AfterIterationContext context, CancellationToken ct)
{
    // Track errors
    var hasErrors = context.ToolResults.Any(r => r.Exception != null);

    context.UpdateMiddlewareState<ErrorTrackingState>(state =>
        hasErrors
            ? state with { ConsecutiveFailures = state.ConsecutiveFailures + 1 }
            : state with { ConsecutiveFailures = 0 }
    );

    return Task.CompletedTask;
}

Function Level Hooks

BeforeParallelBatchAsync

csharp

Task BeforeParallelBatchAsync(
    BeforeParallelBatchContext context,
    CancellationToken cancellationToken)

When: Before parallel functions execute (once per batch) Not called for single function execution Context Properties:

ParallelFunctions - List of functions about to run in parallel
RunOptions - Read-only user options

Use Cases: Batch permissions, resource reservation, batch validation

Example:

csharp

public async Task BeforeParallelBatchAsync(
    BeforeParallelBatchContext context,
    CancellationToken ct)
{
    var functions = context.ParallelFunctions.Select(f => f.Name ?? "_unknown").ToList();
    var approved = await _permissions.RequestBatchApproval(functions, ct);

    // Store in state for BeforeFunctionAsync to check
    context.UpdateMiddlewareState<BatchApprovalsState>(_ => new BatchApprovalsState
    {
        ApprovedFunctions = approved
    });
}

BeforeFunctionAsync

csharp

Task BeforeFunctionAsync(
    BeforeFunctionContext context,
    CancellationToken cancellationToken)

When: Before each function executes Context Properties:

Function - AIFunction being called
FunctionCallId - Unique ID for this invocation
Arguments - Function arguments (read-only dictionary)
ToolkitName, SkillName - Optional context
RunOptions - Read-only user options

Control Flow:

Set BlockExecution = true to prevent execution
Set OverrideResult when blocking to provide custom result

Use Cases: Permission checks, argument validation, logging

Example:

csharp

public Task BeforeFunctionAsync(BeforeFunctionContext context, CancellationToken ct)
{
    // Check batch approvals from BeforeParallelBatchAsync
    var isApproved = context.GetMiddlewareState<BatchApprovalsState>()?
        .ApprovedFunctions?.Contains(context.Function.Name) ?? false;

    if (!isApproved)
    {
        context.BlockExecution = true;
        context.OverrideResult = "User denied permission";
    }

    return Task.CompletedTask;
}

WrapFunctionCallAsync

csharp

Task<object?> WrapFunctionCallAsync(
    FunctionRequest request,
    Func<FunctionRequest, Task<object?>> handler,
    CancellationToken cancellationToken)

When: Wraps function execution Request Properties (Immutable):

Function - AIFunction
CallId - Unique ID
Arguments - Read-only dictionary
State - Agent state
ToolkitName, SkillName - Optional context

Use Cases: Retry, timeout, caching, result transformation

Example - Timeout:

csharp

public async Task<object?> WrapFunctionCallAsync(
    FunctionRequest request,
    Func<FunctionRequest, Task<object?>> handler,
    CancellationToken ct)
{
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
    cts.CancelAfter(TimeSpan.FromSeconds(30));

    try
    {
        return await handler(request);
    }
    catch (OperationCanceledException)
    {
        throw new TimeoutException($"Function {request.Function.Name} timed out");
    }
}

AfterFunctionAsync

csharp

Task AfterFunctionAsync(
    AfterFunctionContext context,
    CancellationToken cancellationToken)

When: After function completes Context Properties:

Function - AIFunction that executed
FunctionCallId - Call ID
Result - Function result (or custom result if blocked)
Exception - Exception thrown (null if successful)
RunOptions - Read-only user options

Always runs - Even if function failed

Use Cases: Logging, result transformation, error handling

Example:

csharp

public Task AfterFunctionAsync(AfterFunctionContext context, CancellationToken ct)
{
    if (context.Exception != null)
    {
        _logger.LogError(
            "Function {Name} failed: {Error}",
            context.Function.Name,
            context.Exception.Message
        );
    }

    return Task.CompletedTask;
}

Error Handling

OnErrorAsync (Centralized Error Handling)

csharp

Task OnErrorAsync(
    ErrorContext context,
    CancellationToken cancellationToken)

When: ANY error occurs during execution Context Properties:

Error - Exception that occurred
Source - Where error originated (enum):
- ModelCall - Error during LLM call
- ToolCall - Error during tool execution
- Iteration - Error during iteration
- MessageTurn - Error during message turn
Iteration - Current iteration (if applicable)

Execution: Runs in REVERSE order (like After* hooks)

Use Cases: Centralized logging, circuit breakers, error transformation

Example - Circuit Breaker:

csharp

public Task OnErrorAsync(ErrorContext context, CancellationToken ct)
{
    if (context.Source == ErrorSource.ToolCall)
    {
        // Increment failure count
        context.UpdateMiddlewareState<ErrorTrackingState>(s => s.IncrementFailures());

        // Check if we should terminate
        var failures = context.GetMiddlewareState<ErrorTrackingState>()?.ConsecutiveFailures ?? 0;
        if (failures >= 3)
        {
            context.UpdateState(s => s with
            {
                IsTerminated = true,
                TerminationReason = "Circuit breaker: too many errors"
            });
        }
    }

    return Task.CompletedTask;
}

State Management

Reading Middleware State

Use context.GetMiddlewareState<T>() for simple reads:

csharp

var count = context.GetMiddlewareState<MyCustomState>()?.Count ?? 0;

// For reading core state, use Analyze
var isTerminated = context.Analyze(s => s.IsTerminated);

Updating Middleware State

Updates are immediate (visible to subsequent hooks):

csharp

// Simple middleware state update
context.UpdateMiddlewareState<MyCustomState>(s => s with { Count = s.Count + 1 });

// Advanced: Update middleware state + core state atomically
context.UpdateState(s =>
{
    var state = s.MiddlewareState.MyCustomState ?? new();
    return s with
    {
        MiddlewareState = s.MiddlewareState.WithMyCustomState(updatedState),
        IsTerminated = true
    };
});

// Next hook sees updated state immediately!
var newValue = context.GetMiddlewareState<MyCustomState>();

Updates are immediate - no scheduled updates or pending state.

See 04.2 Middleware State for details.

Events

Emit One-Way Event

csharp

context.Emit(new TextDeltaEvent { Text = "Processing..." });

Request/Response Pattern

csharp

var request = new PermissionRequestEvent
{
    FunctionName = context.Function.Name,
    RequestId = Guid.NewGuid().ToString()
};

context.Emit(request);

var response = await context.WaitForResponseAsync<PermissionResponse>(
    request.RequestId,
    cancellationToken
);

if (!response.Approved)
{
    context.BlockExecution = true;
}

See 05.3 Middleware Events for details.

Context Property Quick Reference

Property	Type	Available In	Mutable
`AgentName`	`string`	All hooks	No
`ConversationId`	`string?`	All hooks	No
`State`	`AgentLoopState`	All hooks	Via `UpdateState()`
`UserMessage`	`ChatMessage`	BeforeMessageTurn+	No
`ConversationHistory`	`List<ChatMessage>`	BeforeMessageTurn+	Yes
`FinalResponse`	`ChatMessage`	AfterMessageTurn	No
`TurnHistory`	`List<ChatMessage>`	AfterMessageTurn	Yes
`Iteration`	`int`	BeforeIteration+	No
`Messages`	`List<ChatMessage>`	BeforeIteration	Yes
`Options`	`ChatOptions`	BeforeIteration	Yes
`Response`	`ChatMessage`	BeforeToolExecution+	No
`ToolCalls`	`IReadOnlyList<FunctionCallContent>`	BeforeToolExecution+	No
`ToolResults`	`IReadOnlyList<FunctionResultContent>`	AfterIteration	No
`ParallelFunctions`	`IReadOnlyList<AIFunction>`	BeforeParallelBatch	No
`Function`	`AIFunction`	BeforeFunction+	No
`FunctionCallId`	`string`	BeforeFunction+	No
`Arguments`	`IReadOnlyDictionary`	BeforeFunction+	No
`Result`	`object?`	AfterFunction	No
`Exception`	`Exception?`	AfterFunction	No
`Error`	`Exception`	OnErrorAsync	No
`Source`	`ErrorSource`	OnErrorAsync	No

Next Steps

05.2 Middleware State - Manage typed state
05.3 Middleware Events - Emit events and handle responses
05.4 Built-in Middleware - Ready-to-use middleware
05.5 Custom Middleware - Build your own

Middleware Lifecycle ​

Execution Flow ​

Execution Order ​

Onion Architecture (Wrap Hooks) ​

Turn Level Hooks ​

BeforeMessageTurnAsync ​

AfterMessageTurnAsync ​

Iteration Level Hooks ​

BeforeIterationAsync ​

WrapModelCallAsync (Simple Pattern - Recommended) ​

WrapModelCallStreamingAsync (Advanced Pattern) ​

BeforeToolExecutionAsync ​

AfterIterationAsync ​

Function Level Hooks ​

BeforeParallelBatchAsync ​

BeforeFunctionAsync ​

WrapFunctionCallAsync ​

AfterFunctionAsync ​

Error Handling ​

OnErrorAsync (Centralized Error Handling) ​

State Management ​

Reading Middleware State ​

Updating Middleware State ​

Events ​

Emit One-Way Event ​

Request/Response Pattern ​

Context Property Quick Reference ​

Next Steps ​

Middleware Lifecycle

Execution Flow

Execution Order

Onion Architecture (Wrap Hooks)

Turn Level Hooks

BeforeMessageTurnAsync

AfterMessageTurnAsync

Iteration Level Hooks

BeforeIterationAsync

WrapModelCallAsync (Simple Pattern - Recommended)

WrapModelCallStreamingAsync (Advanced Pattern)

BeforeToolExecutionAsync

AfterIterationAsync

Function Level Hooks

BeforeParallelBatchAsync

BeforeFunctionAsync

WrapFunctionCallAsync

AfterFunctionAsync

Error Handling

OnErrorAsync (Centralized Error Handling)

State Management

Reading Middleware State

Updating Middleware State

Events

Emit One-Way Event

Request/Response Pattern

Context Property Quick Reference

Next Steps