Ripple: Profile-Guided Instruction Cache Replacement for Data Center Applications


Modern data center applications exhibit deep software stacks yielding large instruction footprints that frequently lead to instruction cache (I-cache) misses degrading performance, cost-efficiency, and energy efficiency. Although numerous mechanisms have been proposed to mitigate instruction cache misses, they still fall short of ideal cache behavior, and furthermore, introduce significant hardware overheads. We first investigate why existing I-cache miss mitigation mechanisms achieve sub-optimal performance for data center applications. We find that widely-studied instruction prefetches fall short due to wasteful prefetch-induced evictions that are not handled by existing replacement policies. Alas, existing replacement policies are unable to mitigate wasteful evictions since they lack complete knowledge of a data center application's complex program behavior.

To make existing replacement policies aware of these eviction-inducing program behaviors, we propose Ripple, a novel software-only technique that profiles programs and uses program context to inform the underlying replacement policy about efficient replacement decisions. Ripple carefully identifies program contexts that lead to I-cache misses and sparingly injects cache line eviction instructions in suitable program locations at link time. We evaluate Ripple using nine popular data center applications and demonstrate that Ripple enables any replacement policy to achieve speedup that is closer to that of an ideal I-cache. Specifically, Ripple achieves an average performance improvement of 1.6% (up to 2.13%) due to a mean 19% (up to 28.6%) I-cache miss reduction.

In proceedings of the 48th International Symposium on Computer Architecture (Acceptance rate: 76/406 = 18.7%)
Akshitha Sriraman
Akshitha Sriraman
Assistant Professor

I am an Assistant Professor in the Department of Electrical and Computer Engineering at Carnegie Mellon University. My research bridges computer architecture and software systems, with a focus on making hyperscale data center systems more efficient (via solutions that span the systems stack). The central theme of my work is to (1) design software that is aware of new hardware constraints and (2) architect hardware that efficiently supports new hyperscale software requirements.