Awide Data Processor (ADP)

Intelligent Key-Value Caching

ADP KV Cache Offloading transforms multi-GPU systems from memory-limited clusters into scalable, compute-efficient AI infrastructure.

Download Data Sheet

Delivers 4X Higher Effective System Capacity

4X more concurrent users under SLA

4X higher sustainable throughput

Linear scaling preserved across multi-GPU system

Throughput vs Concurrent Clients

GPU + ADP
GPU-only
At SLA boundary: GPU-only: 16 clients / 3.47 req/s | GPU + ADP: 64 clients / 12.27 req/s

Eliminates Multi-GPU Memory Bottlenecks

The system transitions from:

Without ADP
  • Memory fragmentation across GPUs
  • KV cache eviction under load
  • Inter-GPU inefficiencies
With ADP
  • Unified KV cache via NVMe (ADP RAID)
  • Stable latency across all GPUs
  • Efficient utilization of full GPU cluster

Enables Massive Infrastructure Consolidation

Equivalent of +1 additional server with 8xGPUs worth of memory capacity
Avoidance of One GPU Server CapEx
2X density improvement per rack
Up to 50% energy savings vs GPU-only equivalent scaling

Strengthens SLA at Scale

Under heavy load: TPOT ≤50 ms maintained up to 64 clients

TPOT vs Concurrent Clients

50ms SLA Threshold

GPU + ADP
GPU-only
SLA 50ms

TTFT vs Concurrent Clients

Stability Under Load

GPU + ADP
GPU-only

Stable TTFT across concurrency range

No recompute storms

No KV eviction collapse

Strategic Enterprise Impact

Enterprise-grade
multi-GPU stability

ADP turns multi-GPU inference from a memory-bound experiment into a production platform — with predictable scaling, lower infrastructure cost, and consistent SLA performance at any concurrency.

High-density AI inference clusters

Pack more inference capacity into every rack without sacrificing throughput or stability.

Predictable scaling across GPU fleets

Stable latency and consistent throughput as concurrency grows — no eviction storms, no recompute collapses.

Reduced GPU dependency for memory scaling

Offload KV cache to NVMe through ADP — scale memory capacity without adding GPU servers.

Lower power, cooling, and rack footprint

Up to 50% energy reduction and one full server saved versus GPU-only scaling for the same workload.

Significantly improved performance-per-dollar

4X throughput at the SLA boundary and up to 50% CapEx reduction — the best economics for high-memory AI inference workloads.

Key Metrics Results

Metric GPU-Only GPU + ADP Improvement
Max Sustainable Clients* 16 64 4X
Sustainable Throughput 3.47 req/s 12.27 req/s ~4X
TTFT at Sustainable Point ~245 ms ~82 ms 3X faster
KV Cache Pool Required 966.86 GiB 966.86 GiB** GPU avoided
GPUs Required for Equivalent Memory 2 servers with 8xGPU 1 server with 8xGPU 1 server with 8xGPU

* TPOT ≤50ms  |  ** NVMe-backed

4X more users per cluster
4X throughput per deployment
Stable latency at scale
Elimination of GPU-memory bottleneck
Saving one server with 8xGPU

Financial Impact

Lower CapEx, smaller fleets, and better performance-per-dollar for high-memory AI inference workloads.

CapEx Reduction

Cut hardware spend by avoiding a second GPU server while delivering the same memory-intensive inference workload.

  • Up to 50% infrastructure CapEx reduction
  • One full 8xGPU server saved per cluster
  • Faster ROI for new inference deployments

Lower Expansion Cost

Scale inference environments without proportionally scaling GPU count — extend memory capacity through ADP and NVMe.

  • Add capacity without adding GPU servers
  • Scale memory cheaply via NVMe-backed cache
  • Predictable cost-per-user as fleets grow

Performance-per-Dollar Leadership

Deliver 4X more sustainable throughput on the same hardware budget for high-memory AI inference workloads.

  • 4X more concurrent users at SLA
  • ~4X higher sustainable throughput
  • Lower OpEx through energy & cooling savings

Data Center Rack Density Impact

2X higher density per rack
Up to 2X more inference capacity per rack
Up to 2X AI compute density expansion → more AI capacity per rack
Reduced cooling requirements
Reduced PDU requirements
Lower data center footprint

GPU-only: 2 servers (8xGPU) = 14U  |  ADP: 1 server (8xGPU) = 7U

Operational Expenditure Reduction

Up to 50% energy reduction
Lower cooling demand
Lower PDU load
Lower rack power density for the same workload
Improved data center efficiency per deployed inference workload

GPU-only requires 2 servers  |  ADP requires 1 server  |  Each server ~10-12 kW under load

Supported Models

Every model passes the same regression on hardware: each KV page restored from ADP is compared byte for byte with what the engine wrote.

Model Attention Architecture vLLM* MTP
DeepSeek-V4-Flash Compressed sparse MLA + sliding window 0.29.0 —
Kimi K3 Kimi Delta Attention + MLA 0.29.0 —
Kimi-Linear-48B-A3B Kimi Delta Attention + MLA 0.29.0 —
GLM-5.3 FP8 Sparse MLA (DSA) 0.28.0 —
MiniMax M3 MXFP8 Sparse attention with indexer, MoE 0.28.0 —
GLM-5.2 FP8 Sparse MLA (DSA) 0.26 Yes
GLM-4.7-Flash MLA, AWQ 0.26 —
DeepSeek-V2-Lite MLA 0.26 —
Gemma 4 12B Full + sliding-window attention 0.26 Yes
Qwen 3.8 27B Linear + full attention 0.26 Yes
Qwen 3.6 35B-A3B Gated DeltaNet, MoE 0.26 Yes
Qwen 3.6 27B Mamba hybrid, three cache groups 0.26 —
Nemotron 3 Nano 30B-A3B Mamba-2 + sparse attention 0.26 —

* The version each model is verified on and the one we recommend for it  |  One connector code base for vLLM 0.26, 0.28 and 0.29

Go deeper

Deep Dive: The Idea Explained

The full engineering rationale, measurements and methodology behind KV-cache offloading — with raw benchmark data, the cost model and the test stack.

Deep Dive: The Idea Explained