Coordinating Fast Concurrency Adapting with Autoscaling for SLO-Oriented Web Applications

Jianshu Liu, Shungeng Zhang, Qingyang Wang, Jinpeng Wei

Research output: Contribution to journalArticlepeer-review

Abstract

Cloud providers tend to support dynamic computing resources reallocation (e.g., Autoscaling) to handle the bursty workload for web applications (e.g., e-commerce) in the cloud environment. Nevertheless, we demonstrate that directly scaling a bottleneck server without quickly adjusting its soft resources (e.g., server threads and database connections) can cause significant response time fluctuations of the target web application. Since soft resources determine the request processing concurrency of each server in the system, simply scaling out/in the bottleneck service can unintentionally change the concurrency level of related services, inducing either under- or over-utilization of the critical hardware resource. In this paper, we propose the Scatter-Concurrency-Throughput (SCT) model, which can rapidly identify the near-optimal soft resource allocation of each server in the system using the measurement of each server's real-time throughput and concurrency. Furthermore, we implement a Concurrency-aware autoScaling (ConScale) framework that integrates the SCT model to quickly reallocate the soft resources of the key servers in the system to best utilize the new hardware resource capacity after the system scaling. Based on extensive experimental comparisons with two widely used hardware-only scaling mechanisms for web applications: EC2-AutoScaling (VM-based autoscaler) and Kubernetes HPA (container-based autoscaler), we show that ConScale can successfully mitigate the response time fluctuations over the system scaling phase in both VM-based and container-based environments.

Original languageEnglish (US)
JournalIEEE Transactions on Parallel and Distributed Systems
DOIs
StateAccepted/In press - 2022
Externally publishedYes

Keywords

  • Auto-scaling
  • Cloud-based applications
  • Concurrent computing
  • Hardware
  • Resource management
  • Scalability
  • Servers
  • Soft resource
  • Throughput
  • Time factors
  • Topology

ASJC Scopus subject areas

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'Coordinating Fast Concurrency Adapting with Autoscaling for SLO-Oriented Web Applications'. Together they form a unique fingerprint.

Cite this