Built-in Middleware

Ready-to-use middleware for common agent patterns.

Overview

HPD-Agent includes production-ready middleware for:

• Error handling - Circuit breakers, error tracking
• Safety - PII redaction, guardrails
• Optimization - History reduction
• Observability - Logging, telemetry
• Multimodal - Asset storage and management

Circuit Breaker

Prevents infinite loops by detecting repeated identical function calls.

Usage

var agent = new AgentBuilder()
    .WithMiddleware(new CircuitBreakerMiddleware
    {
        MaxConsecutiveCalls = 3 // Default: 3
    })
    .Build();

Behavior

Triggers when a tool is called with identical arguments more than MaxConsecutiveCalls times consecutively.

When triggered:

1. Sets SkipToolExecution = true
2. Emits TextDeltaEvent for user visibility
3. Emits CircuitBreakerTriggeredEvent for telemetry
4. Terminates agent gracefully

Example

// User: "Search for Paris weather"
// LLM: SearchWeb("Paris weather")
// LLM: SearchWeb("Paris weather") // Same call
// LLM: SearchWeb("Paris weather") // Third time
// 🚫 Circuit breaker triggers - prevents 4th call

Configuration

new CircuitBreakerMiddleware
{
    MaxConsecutiveCalls = 5, // Allow up to 5 identical calls
    TerminationMessageTemplate = "  Stopped '{toolName}' after {count} identical calls"
}

---

Error Tracking

Tracks consecutive failures across iterations.

Usage

var agent = new AgentBuilder()
    .WithMiddleware(new ErrorTrackingMiddleware
    {
        MaxConsecutiveErrors = 3 // Default: 3
    })
    .Build();

Behavior

Counts iterations where at least one tool failed. Resets count on successful iteration.

When threshold reached:

1. Sets IsTerminated = true
2. Sets TerminationReason
3. Agent stops gracefully

State

Stores ErrorTrackingStateData in middleware state:

public sealed record ErrorTrackingStateData
{
    public int ConsecutiveFailures { get; init; }
}

Example

// Iteration 1: Tool fails → ConsecutiveFailures = 1
// Iteration 2: Tool fails → ConsecutiveFailures = 2
// Iteration 3: Tool succeeds → ConsecutiveFailures = 0 (reset)
// Iteration 4: Tool fails → ConsecutiveFailures = 1
// Iteration 5: Tool fails → ConsecutiveFailures = 2
// Iteration 6: Tool fails → ConsecutiveFailures = 3 → Terminates

---

Total Error Threshold

Limits total errors across entire conversation turn.

Usage

var agent = new AgentBuilder()
    .WithMiddleware(new TotalErrorThresholdIterationMiddleware
    {
        MaxTotalErrors = 5 // Default: 5
    })
    .Build();

Behavior

Terminates when cumulative tool failures in a turn exceed threshold, regardless of successes between them.

Difference from ErrorTrackingMiddleware:

• ErrorTracking: Resets on success (3 consecutive failures)
• TotalError: Never resets (5 total failures)

Example

// Iteration 1: Tool fails → Total = 1
// Iteration 2: Tool succeeds → Total = 1 (doesn't reset)
// Iteration 3: Tool fails → Total = 2
// ...
// Iteration N: Tool fails → Total = 5 → Terminates

---

History Reduction

Reduces message history to stay under token limits.

Usage

var agent = new AgentBuilder()
    .WithMiddleware(new HistoryReductionMiddleware
    {
        MaxHistoryTokens = 4000, // Default: 4000
        KeepRecentMessages = 5   // Default: 5
    })
    .Build();

Behavior

Runs in BeforeIterationAsync:

1. Estimates token count in context.Messages
2. If over MaxHistoryTokens:
   • Keeps first message (system prompt)
   • Keeps last KeepRecentMessages messages
   • Inserts summary: "[{count} messages summarized to save tokens]"
   • Removes middle messages

Token Estimation

Uses fast approximation: text.Length / 4

For accurate counting, override:

public class AccurateHistoryReductionMiddleware : HistoryReductionMiddleware
{
    protected override int EstimateTokens(string text)
    {
        return _tokenizer.Encode(text).Count; // Use actual tokenizer
    }
}

Example

// Before (6000 tokens):
// [System] You are a helpful assistant
// [User] Message 1
// [Assistant] Response 1
// [User] Message 2
// [Assistant] Response 2
// [User] Message 3
// [Assistant] Response 3
// [User] Current message

// After reduction (KeepRecentMessages = 2):
// [System] You are a helpful assistant
// [System] [5 messages summarized to save tokens]
// [Assistant] Response 3
// [User] Current message

---

PII Middleware

Redacts personally identifiable information from messages.

Usage

var agent = new AgentBuilder()
    .WithMiddleware(new PIIMiddleware
    {
        RedactEmails = true,
        RedactPhones = true,
        RedactSSN = true,
        Placeholder = "[REDACTED]" // Default: "[REDACTED]"
    })
    .Build();

Behavior

Runs in BeforeIterationAsync, scanning context.Messages for:

• Email addresses
• Phone numbers
• Social Security Numbers
• Credit card numbers (if enabled)

Replaces matches with Placeholder.

Configuration

new PIIMiddleware
{
    RedactEmails = true,
    RedactPhones = true,
    RedactSSN = true,
    RedactCreditCards = false, // Default: false
    Placeholder = "***"
}

Example

// Input:
// "My email is john@example.com and phone is 555-1234"

// Output:
// "My email is [REDACTED] and phone is [REDACTED]"

---

Container Middleware

Scopes middleware to specific Toolkits/skills.

Usage

var agent = new AgentBuilder()
    .WithMiddleware(new ContainerMiddleware
    {
        ToolkitName = "WebSearch",
        Middleware = new RetryMiddleware()
    })
    .Build();

Behavior

Wraps another middleware, only executing it when:

• context.ToolkitName matches ToolkitName, OR
• context.SkillName matches SkillName

Use for Toolkit/skill-specific behavior.

Example

// Only retry web search tools
new ContainerMiddleware
{
    ToolkitName = "WebSearch",
    Middleware = new RetryMiddleware { MaxRetries = 5 }
}

// Only log database operations
new ContainerMiddleware
{
    ToolkitName = "Database",
    Middleware = new LoggingMiddleware()
}

---

Document Handling Middleware

Processes document-related operations.

Usage

var agent = new AgentBuilder()
    .WithMiddleware(new DocumentHandlingMiddleware())
    .Build();

Behavior

Intercepts document operations and processes them according to agent configuration.

Note: Marked as legacy - use Skills for document handling in new code.

---

Asset Upload Middleware

Automatically uploads binary assets (images, audio, PDFs) to storage and transforms DataContent → UriContent references.

Usage

Auto-registered - No configuration needed! This middleware is automatically added by AgentBuilder when you have a session store with asset support.

// AssetUploadMiddleware is automatically registered
var store = new JsonSessionStore("./data"); // Has built-in LocalFileAssetStore
var session = await store.LoadOrCreateSessionAsync("session-id");

var agent = await new AgentBuilder()
    .WithProvider("openai", "gpt-4o")
    .Build();

// Add multimodal message with image
session.AddMessage(new ChatMessage(ChatRole.User, [
    new TextContent("What's in this image?"),
    new DataContent(imageBytes, "image/png")
]));

// AssetUploadMiddleware automatically:
// 1. Uploads imageBytes to store.AssetStore
// 2. Replaces DataContent with UriContent (asset://abc123)
// 3. Emits AssetUploadedEvent
await foreach (var evt in agent.RunAsync([], session))
{
    // Handle events
}

// Session now contains UriContent reference, not raw bytes
await session.SaveAsync(); // Saves asset:// URI, not binary data

Behavior

Runs in BeforeIterationAsync:

1. Checks if session.Store.AssetStore exists (zero-cost exit if null)
2. Scans messages for DataContent with binary data
3. Uploads each asset to session.Store.AssetStore
4. Replaces DataContent with UriContent using asset:// URI scheme
5. Updates both context messages (for LLM) AND session messages (for persistence)
6. Emits AssetUploadedEvent or AssetUploadFailedEvent

Zero-Cost Abstraction

If session.Store.AssetStore is null, the middleware returns immediately with zero overhead:

// No AssetStore - middleware does nothing (zero cost)
var inMemoryStore = new InMemorySessionStore(); // AssetStore is null
var session = await inMemoryStore.LoadOrCreateSessionAsync("id");

// DataContent passes through unchanged
session.AddMessage(new ChatMessage(ChatRole.User, [
    new DataContent(imageBytes, "image/png")
]));

Supported Asset Types

Any binary content via DataContent:

• Images: PNG, JPEG, GIF, WebP
• Audio: MP3, WAV, OGG
• Documents: PDF, DOCX
• Videos: MP4, WebM
• Any other: application/octet-stream

Events

AssetUploadedEvent:

public record AssetUploadedEvent(
    string AssetId,
    string MediaType,
    int SizeBytes
) : AgentEvent;

AssetUploadFailedEvent:

public record AssetUploadFailedEvent(
    string MediaType,
    string Error
) : AgentEvent;

Example: Vision Model

var store = new JsonSessionStore("./data");
var session = await store.LoadOrCreateSessionAsync("vision-chat");

var agent = await new AgentBuilder()
    .WithProvider("openai", "gpt-4o") // Vision model
    .Build();

// Add image from file
var imageBytes = await File.ReadAllBytesAsync("photo.jpg");
session.AddMessage(new ChatMessage(ChatRole.User, [
    new TextContent("What's in this photo?"),
    new DataContent(imageBytes, "image/jpeg")
]));

// Asset automatically uploaded, message transformed
await foreach (var evt in agent.RunAsync([], session))
{
    if (evt is AssetUploadedEvent upload)
        Console.WriteLine($"Uploaded {upload.MediaType}: {upload.AssetId}");
}

// Session saved with asset:// reference (not 5MB of binary data)
await session.SaveAsync();

Example: Multiple Assets

session.AddMessage(new ChatMessage(ChatRole.User, [
    new TextContent("Compare these:"),
    new DataContent(image1, "image/png"),
    new TextContent("versus"),
    new DataContent(image2, "image/jpeg"),
    new TextContent("and this PDF:"),
    new DataContent(pdfBytes, "application/pdf")
]));

// All three assets uploaded automatically
await foreach (var evt in agent.RunAsync([], session))
{
    if (evt is AssetUploadedEvent upload)
        Console.WriteLine($"Asset {upload.AssetId}: {upload.SizeBytes} bytes");
}

Session Persistence

The key benefit: sessions store URI references instead of binary data.

Before transformation (in-memory):

{
  "role": "user",
  "contents": [
    { "type": "text", "text": "What's in this image?" },
    { "type": "data", "data": "iVBORw0KG...", "mediaType": "image/png" }
  ]
}

After transformation (persisted):

{
  "role": "user",
  "contents": [
    { "type": "text", "text": "What's in this image?" },
    { "type": "uri", "uri": "asset://abc123", "mediaType": "image/png" }
  ]
}

The asset is stored separately in ./data/assets/abc123.png.

Custom Asset Store

Implement IAssetStore for custom storage (S3, Azure Blob, database):

public class S3AssetStore : IAssetStore
{
    public async Task<string> UploadAssetAsync(
        byte[] data,
        string contentType,
        CancellationToken ct = default)
    {
        var assetId = Guid.NewGuid().ToString("N");
        await _s3Client.PutObjectAsync(new PutObjectRequest
        {
            BucketName = _bucketName,
            Key = assetId,
            InputStream = new MemoryStream(data),
            ContentType = contentType
        }, ct);
        return assetId;
    }

    public async Task<AssetData> DownloadAssetAsync(
        string assetId,
        CancellationToken ct = default)
    {
        var response = await _s3Client.GetObjectAsync(_bucketName, assetId, ct);
        using var ms = new MemoryStream();
        await response.ResponseStream.CopyToAsync(ms, ct);
        return new AssetData(assetId, ms.ToArray(), response.Headers.ContentType);
    }
}

// Use with session store
public class MySessionStore : ISessionStore
{
    public IAssetStore? AssetStore => new S3AssetStore();
    // ... other methods
}

---

Combining Middleware

Stack multiple middleware for layered behavior:

var agent = new AgentBuilder()
    .WithProvider("openai", "gpt-4o")
    .WithTools<MyTools>()
    // Layer 1: Redact PII
    .WithMiddleware(new PIIMiddleware())
    // Layer 2: Reduce history
    .WithMiddleware(new HistoryReductionMiddleware { MaxHistoryTokens = 3000 })
    // Layer 3: Track errors
    .WithMiddleware(new ErrorTrackingMiddleware { MaxConsecutiveErrors = 3 })
    // Layer 4: Circuit breaker
    .WithMiddleware(new CircuitBreakerMiddleware { MaxConsecutiveCalls = 3 })
    .Build();

Execution order:

• Before hooks: PIIMiddleware → HistoryReduction → ErrorTracking → CircuitBreaker
• After hooks: CircuitBreaker → ErrorTracking → HistoryReduction → PIIMiddleware

---

Custom Configuration

All middleware support property-based configuration:

var circuitBreaker = new CircuitBreakerMiddleware
{
    MaxConsecutiveCalls = 5,
    TerminationMessageTemplate = "Custom message: {toolName} called {count} times"
};

var errorTracking = new ErrorTrackingMiddleware
{
    MaxConsecutiveErrors = 2
};

var agent = new AgentBuilder()
    .WithMiddleware(circuitBreaker)
    .WithMiddleware(errorTracking)
    .Build();

---

Built-in Middleware Summary

Middleware	Purpose	Hook	State	Auto-registered
AssetUploadMiddleware	Upload binary assets	BeforeIteration	None	Yes
CircuitBreakerMiddleware	Prevent infinite loops	BeforeToolExecution	CircuitBreakerState	No
ErrorTrackingMiddleware	Track consecutive errors	AfterIteration	ErrorTrackingStateData	No
TotalErrorThresholdIterationMiddleware	Limit total errors	AfterIteration	None	No
HistoryReductionMiddleware	Reduce message history	BeforeIteration	None	No
PIIMiddleware	Redact PII	BeforeIteration	None	No
ContainerMiddleware	Scope middleware	All hooks	None	No
DocumentHandlingMiddleware	Process documents	Various	None (Legacy)	No

---

Next Steps

• 05.1 Middleware Lifecycle - Understand when each hook fires
• 05.2 Middleware State - See how state works in ErrorTracking
• 05.3 Middleware Events - Learn event emission (used in CircuitBreaker)
• 05.5 Custom Middleware - Build your own middleware