Coordinating Fast Concurrency Adapting With Autoscaling for SLO-Oriented Web Applications

Jianshu Liu, Shungeng Zhang, Qingyang Wang, Jinpeng Wei

Research output: Contribution to journalArticlepeer-review


Cloud providers tend to support dynamic computing resources reallocation (e.g., Autoscaling) to handle the bursty workload for web applications (e.g., e-commerce) in the cloud environment. Nevertheless, we demonstrate that directly scaling a bottleneck server without quickly adjusting its soft resources (e.g., server threads and database connections) can cause significant response time fluctuations of the target web application. Since soft resources determine the request processing concurrency of each server in the system, simply scaling out/in the bottleneck service can unintentionally change the concurrency level of related services, inducing either under- or over-utilization of the critical hardware resource. In this paper, we propose the Scatter-Concurrency-Throughput (SCT) model, which can rapidly identify the near-optimal soft resource allocation of each server in the system using the measurement of each server's real-time throughput and concurrency. Furthermore, we implement a Concurrency-aware autoScaling (ConScale) framework that integrates the SCT model to quickly reallocate the soft resources of the key servers in the system to best utilize the new hardware resource capacity after the system scaling. Based on extensive experimental comparisons with two widely used hardware-only scaling mechanisms for web applications: EC2-AutoScaling (VM-based autoscaler) and Kubernetes HPA (container-based autoscaler), we show that ConScale can successfully mitigate the response time fluctuations over the system scaling phase in both VM-based and container-based environments.

Original languageEnglish (US)
Pages (from-to)3349-3362
Number of pages14
JournalIEEE Transactions on Parallel and Distributed Systems
Issue number12
StatePublished - Dec 1 2022


  • Scalability
  • auto-scaling
  • cloud-based applications
  • soft resource

ASJC Scopus subject areas

  • Signal Processing
  • Hardware and Architecture
  • Computational Theory and Mathematics


Dive into the research topics of 'Coordinating Fast Concurrency Adapting With Autoscaling for SLO-Oriented Web Applications'. Together they form a unique fingerprint.

Cite this