BOOST: Concurrent Access to Host Memory and HBM to Accelerate LLM Inference

Anish Saxena

Jae Hyung Ju

Hritvik Taneja

Po-An Tsai

Aamer Jaleel

Christos Kozyrakis Stanford

Moinuddin Qureshi

Preprint, 2026


Abstract

BOOST combines host-memory and HBM bandwidth for LLM inference using wave-aware page placement and runtime memory management. It supports concurrent, bandwidth-proportional access to both memory tiers without changing GPU kernels.