Agile power management for future energy-efficiency servers in data centers running latency-critical applications

When: January 21, 2025
Where: HiPEAC 2025, Barcelona

Online applications running in modern data centers, such as social networks and web search, have moved from a monolithic to a microservice-based architecture, where a monolithic application is decomposed into smaller, interconnected services that communicate explicitly with each other over the network. Due to the increased communication and network overhead, each service must adhere to strict quality-of-service constraints, with 99th percentile latency targets that range from 250 to 500 microseconds. These characteristics render ineffective existing energy-conserving techniques when processors are idle due to the long transition time (order of 100μs) from a deep CPU core idle power state (C-state). Previous research works around this inefficiency through management and scheduling techniques.

This tutorial introduces attendees to agile power-management techniques for future energy-efficient servers. These techniques aim to improve the energy proportionality of server processors running latency-critical applications by tackling the idle-state performance overhead at its root. Specifically, AgileWatts redesigns the Core C-state architecture for latency-critical applications, drastically reducing the transition latency of deep CPU core idle power states while retaining most of their power-saving benefits. AgilePkgC (APC) introduces a new Package C-state that a system can enter once all cores are in a shallow C-state, featuring a nanosecond-scale transition latency.

Organizers and Presenters

• Haris Volos
  Department of Computer Science
  University of Cyprus
• Yanos Sazeides
  Department of Computer Science
  University of Cyprus
• Georgia Antoniou
  Department of Computer Science
  University of Cyprus
• Jawad Haj-Yahya
  Rivos Inc.

Topics

• Current state of the art in server power management and idle power states (C-states)
• Agile Core C-state architecture
• Agile Package C-state architecture
• Role of heterogeneous memory in energy efficiency
• Methodology for evaluating energy-efficient designs with latency-critical applications

Agenda

Reading List

Books

• L. A. Barroso, U. Hölzle , and P. Ranganathan. The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines. Third Edition, Springer, 2019
• C. Gough, I. Steiner, and W. Saunders. CPU Power Management in Energy Efficient Servers: Blueprints for Data Center Optimization. 2015
• J. Haj-Yahya, A. Mendelson, Y. B. Asher, and A. Chattopadhyay. Energy Efficient High Performance Processors: Recent Approaches for Designing Green High Performance Computing. Springer, 2018

Papers

Microservices

• L. Barroso, M. Marty, D. Patterson, and P. Ranganathan. Attack of the killer microseconds. Communications of the ACM, 2017.
• J. Yang, Y. Yue, and K. V. Rashmi. A large scale analysis of hundreds of in-memory cache clusters at Twitter. USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2020
• A. Sriraman and T. F. Wenisch. μSuite: a Benchmark Suite for Microservices. IEEE International Symposium on Workload Characterization (IISWC), 2018.
• N. Dmitry and S.-S. Manfred. On Micro-Services Architecture. International Journal of Open Information Technologies (INJOIT), Vol. 2, No. 9, 2014.
• S. Luo, H. Xu, C. Lu, K. Ye, G. Xu, L. Zhang, Y. Ding, J. He, and C. Xu. Characterizing Microservice Dependency and Performance: Alibaba Trace Analysis. ACM Symposium on Cloud Computing (SoCC), 2021.
• B. Atikoglu, Y. Xu, E. Frachtenberg, S. Jiang, M. Paleczny. Workload Analysis of a Large-Scale Key-Value Store. ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), 2012

Cold-start Latency

• T. Constantinou, Y. Sazeides, P. Michaud, D. Fetis, and A. Seznec. Performance implications of single thread migration on a chip multi-core. ACM SIGARCH Computer Architecture News 33, 4 (Nov. 2005), 80–91. 2005
• D. Schall, A. Margaritov, D. Ustiugov, A. Sandberg, and B. Grot. Lukewarm serverless functions: Characterization and optimization. IEEE/ACM International Symposium on Computer Architecture (ISCA), 2022
• H. Amur, R. Nathuji, M. Ghosh, K. Schwan, and H. Lee. IdlePower: Application-aware management of processor idle states. Workshop on Managed Many-Core Systems (MMCS), 2008
• M. Arora, S. Manne, I. Paul, N. Jayasena, and D. M. Tullsen. Understanding idle behavior and power gating mechanisms in the context of modern benchmarks on CPU-GPU integrated systems. In Proceedings of the 2015 IEEE International Symposium on High Performance Computer Architecture (HPCA), 2015

Power Management – Agile Power Management Techniques

• G. Antoniou, D. B. Bartolini, H. Volos, M. Kleanthous, Z. Wang, K. Kalaitzidis, T. Rollet, Z. Li, O. Mutlu, Y. Sazeides and J. Haj Yahya. Agile C-states: A Core C-state Architecture for Latency Critical Applications Optimizing both Transition and Cold-Start Latency. ACM Transactions on Architecture and Code Optimization (TACO), Vol. 21, Issue 4, 2024.
• J. Haj Yahya, H. Volos, D. B. Bartolini, G. Antoniou, J. S. Kim, Z. Wang, K. Kalaitzidis, T. Rollet, Z. Chen, Y. Geng, O. Mutlu, and Y. Sazeides. AgileWatts: An Energy-Efficient CPU Core Idle-State Architecture for Latency-Sensitive Server. IEEE/ACM International Symposium on Microarchitecture (MICRO). 2022.
• G. Antoniou, H. Volos, D. B. Bartolini, T. Rollet, Y. Sazeides and J. H. Yahya, AgilePkgC: An Agile System Idle State Architecture for Energy Proportional Datacenter Servers. IEEE/ACM International Symposium on Microarchitecture (MICRO). 2022.

Power Management – Hardware Power Management Techniques

• E. A. Burton, G. Schrom, F. Paillet, J. Douglas, W. J. Lambert, K. Radhakrishnan, and M. J. Hill. FIVR - Fully integrated voltage regulators on 4th generation Intel Core SoCs. IEEE Applied Power Electronics Conference and Exposition (APEC), 2014.
• M. Huang, M. Mehalel, R. Arvapalli, and S. He. An Energy Efficient 32-nm 20-MB Shared on-die L3 cache for Intel® Xeon® processor E5 family. IEEE Journal of Solid-State Circuits, Vol. 48, Issue 8, 2013.
• H. Mahmoodi-Meimand and K. Roy. Data-retention Flip-flops for Power-down Applications. IEEE International Symposium on Circuits and Systems, 2004.
• J. Rabinowicz and S. Greenberg, A New Physical Design Flow for a Selective State Retention Based Approach. Journal of Low Power Electronics and Applications, 2021.
• P. Petrica, A. M. Izraelevitz, D. H. Albonesi, and C. A. Shoemaker, Flicker: A Dynamically Adaptive Architecture for Power Limited Multicore Systems. IEEE/ACM International Symposium on Computer Architecture (ISCA), 2013
• J. Haj-Yahya, M. Alser, J. S. Kim, L. Orosa, E. Rotem, A. Mendelson, A. Chattopadhyay, and O. Mutlu, FlexWatts: A Power-and Workload-Aware Hybrid Power Delivery Network for Energy-Efficient Microprocessors. IEEE/ACM International Symposium on Microarchitecture (MICRO), 2020.
• A. Gendler, E. Knoll, and Y. Sazeides, I-DVFS: Instantaneous Frequency Switch During Dynamic Voltage and Frequency Scaling, IEEE Micro, Vol. 41, Issue 5, 2021.

Power Management – Request Scheduling (DVFS + Idle State)

• C.-H. Chou, L. N. Bhuyan, and D. Wong, μDPM: Dynamic Power Management for the Microsecond Era. IEEE International Symposium on High Performance Computer Architecture (HPCA), 2019.
• E. Asyabi, A. Bestavros, E. Sharafzadeh, and T. Zhu, Peafowl: In-application CPU Scheduling to Reduce Power Consumption of In-memory Key-value Stores. ACM Symposium on Cloud Computing (SoCC), 2020.
• H. Kasture, D. B. Bartolini, N. Beckmann, and D. Sanchez, Rubik: Fast Analytical Power Management for Latency-critical Systems. IEEE/ACM International Symposium on Microarchitecture (MICRO), 2015.