BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference
Preprint, 2026
Abstract
BOOST combines host-memory and HBM bandwidth for LLM inference using wave-aware page placement and runtime memory management. It supports concurrent, bandwidth-proportional access to both memory tiers without changing GPU kernels.