Our review

This pattern provides performance strategies for large datasets: streaming, pagination, optimized counting, and bulk operations.

Strengths

Provides concrete strategies (streaming, bulk, fast count)
Includes quantified benchmarks showing dramatic speedups
Offers a pagination pattern with total count for web APIs
Covers common antipatterns with correct alternatives

Limitations

Specific to the Koan library, not universal
Benchmarks are idealized and may vary in practice
Does not cover caching or indexing aspects

When to use it

Use this pattern when dealing with large datasets and needing to optimize database query performance.

When not to use it

Avoid this pattern for small data volumes or when code simplicity matters more than micro-optimizations.

Examples

Stream large dataset

How can I efficiently process 1 million records from the database without running out of memory? Show me streaming.

Fast count for pagination

I need a fast count of total records for my API pagination. How to get it in milliseconds instead of seconds?

Bulk insert 1000 items

I have 1000 todos to insert efficiently. How to bulk save them in a single operation?

name: koan-performance description: Streaming, pagination, count strategies, bulk operations

Koan Performance

Core Principle

Optimize for scale from day one. Use streaming for large datasets, batch operations for bulk changes, fast counts for UI, and pagination for web APIs.

Performance Patterns

Streaming (Large Datasets)

// ❌ WRONG: Load everything into memory
var allTodos = await Todo.All(); // 1 million records!

// ✅ CORRECT: Stream in batches
await foreach (var todo in Todo.AllStream(batchSize: 1000))
{
    await ProcessTodo(todo);
}

Count Strategies

// Fast count (metadata estimate - 1000x+ faster)
var fast = await Todo.Count.Fast(ct); // ~5ms for 10M rows

// Exact count (guaranteed accuracy)
var exact = await Todo.Count.Exact(ct); // ~25s for 10M rows

// Optimized (framework chooses)
var optimized = await Todo.Count; // Uses Fast if available

Use Fast for: Pagination UI, dashboards, estimates Use Exact for: Critical business logic, reports, inventory

Bulk Operations

// Bulk create
var todos = Enumerable.Range(1, 1000)
    .Select(i => new Todo { Title = $"Task {i}" })
    .ToList();
await todos.Save(); // Single operation

// Bulk removal
await Todo.RemoveAll(RemoveStrategy.Fast); // TRUNCATE/DROP (225x faster)

Batch Retrieval

// ❌ WRONG: N queries
foreach (var id in ids)
{
    var todo = await Todo.Get(id);
}

// ✅ CORRECT: 1 query
var todos = await Todo.Get(ids);

Pagination

public async Task<IActionResult> GetTodos(
    int page = 1,
    int pageSize = 20,
    CancellationToken ct = default)
{
    var result = await Todo.QueryWithCount(
        t => !t.Completed,
        new DataQueryOptions { OrderBy = nameof(Todo.Created), Descending = true },
        ct);

    Response.Headers["X-Total-Count"] = result.TotalCount.ToString();
    return Ok(result.Items);
}

Performance Benchmarks

| Operation | Inefficient | Efficient | Speedup | |-----------|-------------|-----------|---------| | Bulk Remove (1M) | DELETE loop ~45s | TRUNCATE ~200ms | 225x | | Count (10M) | Full scan ~25s | Metadata ~5ms | 5000x | | Batch Get (100) | 100 queries | 1 query | 100x | | Stream (1M) | Load all (OOM) | Stream batches | Memory safe |

When This Skill Applies

✅ Performance tuning
✅ Large datasets
✅ Optimization
✅ Production readiness
✅ Memory issues
✅ Query optimization

Reference Documentation

Example Code: .claude/skills/entity-first/examples/batch-operations.cs
Guide: docs/guides/performance.md
Sample: samples/S14.AdapterBench/ (Performance benchmarks)

Koan Performance Optimization

Recommended for