Agile power management for future energy-efficiency servers in data centers running latency-critical applications

When: January 21, 2025
Where: HiPEAC 2025, Barcelona

Online applications running in modern data centers, such as social networks and web search, have moved from a monolithic to a microservice-based architecture, where a monolithic application is decomposed into smaller, interconnected services that communicate explicitly with each other over the network. Due to the increased communication and network overhead, each service must adhere to strict quality-of-service constraints, with 99th percentile latency targets that range from 250 to 500 microseconds. These characteristics render ineffective existing energy-conserving techniques when processors are idle due to the long transition time (order of 100μs) from a deep CPU core idle power state (C-state). Previous research works around this inefficiency through management and scheduling techniques.

This tutorial introduces attendees to agile power-management techniques for future energy-efficient servers. These techniques aim to improve the energy proportionality of server processors running latency-critical applications by tackling the idle-state performance overhead at its root. Specifically, AgileWatts redesigns the Core C-state architecture for latency-critical applications, drastically reducing the transition latency of deep CPU core idle power states while retaining most of their power-saving benefits. AgilePkgC (APC) introduces a new Package C-state that a system can enter once all cores are in a shallow C-state, featuring a nanosecond-scale transition latency.

Organizers and Presenters

• Haris Volos
  Department of Computer Science
  University of Cyprus
• Yanos Sazeides
  Department of Computer Science
  University of Cyprus
• Georgia Antoniou
  Department of Computer Science
  University of Cyprus
• Jawad Haj-Yahya
  Rivos Inc.

Topics

• Current state of the art in server power management and idle power states (C-states)
• Agile Core C-state architecture
• Agile Package C-state architecture
• Role of heterogeneous memory in energy efficiency
• Methodology for evaluating energy-efficient designs with latency-critical applications

Agenda

Reading List

• Georgia Antoniou, Davide B. Bartolini, Haris Volos, Marios Kleanthous, Zhe Wang, Kleovoulos Kalaitzidis, Tom Rollet, Ziwei Li, Onur Mutlu, Yiannakis Sazeides and Jawad Haj Yahya. Agile C-states: A Core C-state Architecture for Latency Critical Applications Optimizing both Transition and Cold-Start Latency. ACM Transactions on Architecture and Code Optimization (TACO). 2024. To appear.
• Jawad Haj Yahya, Haris Volos, Davide B. Bartolini, Georgia Antoniou, Jeremie S. Kim, Zhe Wang, Kleovoulos Kalaitzidis, Tom Rollet, Zhirui Chen, Ye Geng, Onur Mutlu, and Yiannakis Sazeides. AgileWatts: An Energy-Efficient CPU Core Idle-State Architecture for Latency-Sensitive Server. 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). 2022.
• Georgia Antoniou, Haris Volos, Davide B. Bartolini, Tom Rollet, Yiannakis Sazeides and J. H. Yahya, AgilePkgC: An Agile System Idle State Architecture for Energy Proportional Datacenter Servers. 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). 2022.
• Akshitha Sriraman and Thomas F. Wenisch. μSuite: A Benchmark Suite for Microservices. IEEE International Symposium on Workload Characterization (IISWC). 2018.
• C. Gough, I. Steiner, and W. Saunders, “CPU Power Management,” in Energy Efficient Servers: Blueprints for Data Center Optimization, 2015
• M. Arora, S. Manne, I. Paul, N. Jayasena and D. M. Tullsen, Understanding idle behavior and power gating mechanisms in the context of modern benchmarks on CPU-GPU Integrated systems, IEEE 21st International Symposium on High Performance Computer Architecture (HPCA), 2015