New tool enables comprehensive data center performance assessment

An overview of SDCBench. Credit: Intelligent Computing (2022). DOI: 10.34133/2022/9810691

Data centers are used by a myriad of businesses, institutions, and other operations, including large-scale online services such as e-commerce, search engines, online maps, social media , advertising, etc. These data centers co-locate workloads, which involves sharing data center resources to improve server utilization.

However, this can cause performance degradation. To investigate this problem and find potential solutions, researchers need to have workload co-location assessment tools. Although such tools have been developed before, they may only measure one aspect to the detriment of other factors, which limits their usefulness.

A team of researchers from Tianjin University and Dalian University of Technology, both in China, have now developed a benchmark suite for workload colocation, called SDCBench, to address the previous issues and provide a full analysis.

The research has been published in Intelligent Computing September 7.

“Workload colocation can cause performance interference that can lead to unpredictable performance degradation of cloud services, which not only reduces user experience, but also impairs resource efficiency in data centers” , said corresponding author Laiping Zhao, associate professor at Tianjin Key Lab of Advanced. Networking at the College of Intelligence and Computer Science, Tianjin University, China.

To overcome this problem, researchers are trying to improve the isolation capability – which refers to privacy issues related to resource sharing in data centers – of cloud systems through hardware and software approaches. However, the suggested solutions may require software updates or new hardware, which some cloud providers are unable or unwilling to provide.

“The need for predictable service performance in data centers brings new challenges and opportunities for designing cloud systems that seek to improve resource utilization at the server level but do not hurt performance at the application level,” Zhao said.

“Unfortunately, the lack of a comprehensive suite of workload colocation benchmarks makes it difficult to study this emerging issue. A workload colocation benchmark can help cloud providers understand and improve capabilities infrastructure isolation, increasing adoption by cloud users.”

Researchers developed SDCBench, a benchmark suite for workload colocation that includes 16 latency-critical services and applications (meaning there should be very little lag in response time) and applications that cover a wide range of cloud scenarios.

“SDCBench enables cloud tenants to understand performance isolation capability in data centers and choose their best-suited cloud services,” Zhao said. “For cloud providers, it also helps them improve service quality to increase revenue.”

Along with the introduction of the new suite of benchmarks, the researchers propose the concept of latency entropy, inspired by the physical definition of entropy to designate the degree of disorder within a system, to measure uncertainty cloud systems.

“When a shared resource conflict occurs between different applications, system behaviors become messy and unpredictable,” Zhao said. “To help users understand changes in application performance with observable metrics, SDCBench defines latency entropy which describes variations in tail latency for measuring system isolation capability.”

Researchers demonstrated that SDCBench can simulate different cloud scenarios by collocating workloads with simple configurations. They also evaluated and compared latency entropy across major cloud providers using their benchmark tool.

One of the most exciting things about the research, Zhao says, is that SDC Bench and a full framework based on it are publicly available.

“We have implemented a comprehensive assessment framework based on SDCBench that can automatically configure, deploy and assess applications on cloud platforms, and this framework is open source and can be easily extended to new cloud systems”, Zhao said.

Powered by Intelligent Computing