v0.1.10 Epic: KDN Distribution Infrastructure Contracts and Observability Baseline

[rickisba]

Objective

Deliver the v0.1.10 contracts, state vocabulary, compatibility boundaries, and observability foundation required for CacheRoute to evolve into a network-level KVCache infrastructure.

The long-term architecture is:

KDN = KVCache Distribution Network: infrastructure for maintaining, locating, distributing, replicating, and governing KVCache resources across a network.

KDN follows a CDN-like model for KVCache:

Artifact Producer / Origin
        == KDN Distribution Plane
        == KDN Edge / Target LMCache Runtime

The node-local hot path remains:

vLLM == LMCacheMPConnector == local LMCache MP == local L1 / L2

KDN does not replace LMCache. LMCache owns node-local token/chunk/key/layout/serde, L1/L2, cache-object operations, and physical residency. KDN owns the network-level Artifact directory, Node/Region topology, logical Replica state, compatible source selection, placement/replication intent, TransferSession orchestration, lifecycle, policy, and global observations.

Roadmap and Issue Governance

The stable architecture, diagrams, version route, and acceptance goals are maintained in:

• doc/CacheRoute-v0.2.0-plan.md
• doc/CacheRoute-v0.2.0-plan-CN.md

Those roadmap files intentionally do not contain current Issue/PR completion tables because issue scope and status will continue to change. This Epic is the dynamic source for:

• current sub-Issue mapping;
• completion status;
• delivery order;
• scope corrections caused by implementation findings;
• links to replacement or follow-up Issues.

Runtime Baseline

• v0.1.10-0: Upgrade CacheRoute Serving Baseline to vLLM 0.25.1 and LMCache 0.5.2 #148 established Python 3.12, CUDA 13.0, PyTorch 2.11.0, vLLM 0.25.1, and LMCache 0.5.2 through merged PR Introduce v1 runtime compatibility, modern serving profile, and MP startup path #149 and PR Add a v1 environment entry without changing the existing homepage #151.
• v0.1.10-1: Add Instance Capability Fingerprints and Compatibility-Aware Registration #138 established Instance capability fingerprints and compatibility-aware registration through PR v0.1.10: add instance capability fingerprints #143.
• v1 is the only development line for new functionality.
• legacy remains runnable but feature-frozen.
• auto is startup-only migration discovery and must resolve to one frozen Profile.
• A v1 request cannot silently use a Legacy write path.

KDN Infrastructure Boundary

KDN Control Plane

Owns:

• Artifact global identity, version, and compatibility;
• KDN Node, Region, and topology registry;
• logical Replica directory and desired state;
• Source/Target selection;
• placement, replication, migration, prefetch, pin, purge, and rebuild intent;
• quota, authorization, audit, lifecycle, and policy;
• RouteDecision and DistributionPlan.

KDN Distribution Plane

Will own:

• KDN Node Agent;
• TransferSession;
• Pull, Push, Relay, Resume, Retry, Cancel, and verification;
• bandwidth admission, rate limit, queue, and backpressure;
• publishing transferred data into target LMCache Runtime;
• transfer bytes, token coverage, bandwidth, latency, and failure observations.

LMCache Runtime

Owns:

• Token Database, chunk/hash/key generation;
• physical KV objects and layouts;
• serde and device-specific representation;
• node-local L1/L2 residency and adapter cascade;
• Store, Retrieve, Prefetch, Pin, Unpin, Clear;
• node-local capacity, locking, quota, and eviction execution;
• actual hit-token, remote-read, metrics, and events.

Scope and Current Status

• v0.1.10-0: Upgrade CacheRoute Serving Baseline to vLLM 0.25.1 and LMCache 0.5.2 #148 — serving/runtime baseline.
• v0.1.10-1: Add Instance Capability Fingerprints and Compatibility-Aware Registration #138 — Instance capability fingerprint and registration compatibility.
• v0.1.10-2: Define v1 Runtime, Artifact, LMCache Observation, Operation Task, and Queue State Models #139 — define v1 Runtime, Artifact, KDN Node/Replica observation, operation, transfer-aware task, and queue state vocabulary; retain a read-only Legacy projection.
• v0.1.10-3: Define Versioned KDN Cache Service Facade and LMCache Gateway Contracts #140 — define versioned KDN Control API, Distribution API/Protocol, LMCache Gateway contracts, capability negotiation, and structured fallback.
• v0.1.10-4: Add Unified v1 LMCache Gateway, Tier, Adapter, and Queue Observability #141 — add unified KDN Control/Distribution/LMCache/queue/vLLM observability, including RouteDecision and TransferSession stages.
• v0.1.10-5: Add v1/Legacy Dual-Profile and LMCache Gateway Regression Validation #142 — add v1/Legacy dual-profile, Gateway, state-machine, and distribution-contract regression validation.

Sub-Issue wording may be updated as implementation progresses. No closed PR is treated as the implementation baseline for unfinished work.

Required v0.1.10 Contracts

v0.1.10 freezes the vocabulary needed by later KDN cluster releases without implementing the full multi-node data plane:

• RuntimeProfile;
• LMCacheCompatibilityProfile;
• Instance Capability;
• CacheArtifact;
• KDNNode;
• logical CacheReplicaRecord with Desired/Observed separation;
• CacheOperationTask;
• transfer-aware operation and queue vocabulary;
• QueueWork;
• versioned KDN Control and Distribution contract vocabulary;
• immutable Capability Snapshot and Node Generation;
• stable Trace stages and provenance;
• Legacy Redis compatibility projection.

Release Acceptance Criteria

• Domain objects contain no KV bytes, plaintext credentials, raw Redis keys, LMCache-private classes, or device pointers.
• KDN network-level identity is expressed through Artifact, Replica, Node, Route, Operation, and Transfer vocabulary.
• Desired Replica state and observed physical existence are distinguishable.
• Node-local physical state remains authoritative in LMCache observations.
• Unknown capabilities are never interpreted as supported.
• Control API and bulk-data Distribution Protocol are distinct.
• Transfer operations have idempotency, cancellation, retry, timeout, and structured error vocabulary.
• Predicted, desired, observed, measured, inferred, stale, unsupported, and Legacy-projected values are distinguishable.
• Existing Legacy startup, registration, text injection, Redis KV injection, forwarding, and demos remain available.
• Core tests run in CPU-only CI without a GPU, live vLLM endpoint, Redis service, or LMCache daemon.

Non-Goals for v0.1.10

• Implementing the complete multi-node KDN Distribution Plane.
• Reimplementing LMCache Token Database, StorageManager, serde, locks, or Paged KV.
• Fixing one transport such as Redis, NIXL, Mooncake, RDMA, or object storage as the permanent KDN protocol.
• Moving bulk KV data through the KDN Control Plane.
• Implementing the full Proxy ExecutionGraph scheduler.
• Adding product features to Legacy.

Delivery Order

1. v0.1.10-0: Upgrade CacheRoute Serving Baseline to vLLM 0.25.1 and LMCache 0.5.2 #148 runtime baseline — complete
2. v0.1.10-1: Add Instance Capability Fingerprints and Compatibility-Aware Registration #138 capability fingerprints — complete
3. v0.1.10-2: Define v1 Runtime, Artifact, LMCache Observation, Operation Task, and Queue State Models #139 state, identity, Replica, operation, and queue vocabulary
4. v0.1.10-3: Define Versioned KDN Cache Service Facade and LMCache Gateway Contracts #140 KDN Control/Distribution and LMCache Gateway contracts
5. v0.1.10-4: Add Unified v1 LMCache Gateway, Tier, Adapter, and Queue Observability #141 end-to-end infrastructure observability
6. v0.1.10-5: Add v1/Legacy Dual-Profile and LMCache Gateway Regression Validation #142 dual-profile and contract regression validation

Each subtask must be independently testable and must keep business logic out of demo_*.py.