Mitigating tail response time of n-tier applications: The impact of asynchronous invocations

Qingyang Wang, Shungeng Zhang, Yasuhiko Kanemasa, Calton Pu

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Consistent low response time is essential for e-commerce due to intense competitive pressure. However, practitioners of web applications have often encountered the long-tail response time problem in cloud data centers as the system utilization reaches moderate levels (e.g., 50%). Our fine-grained measurements of an open source n-tier benchmark application (RUBBoS) show such long response times are often caused by Cross-tier Queue Overflow (CTQO). Our experiments reveal the CTQO is primarily created by the synchronous nature of RPC-style call/response inter-tier communications, which create strong inter-tier dependencies due to the request processing chain of classic n-tier applications composed of synchronous RPC/thread-based servers. We remove gradually the dependencies in n-tier applications by replacing the classic synchronous servers (e.g., Apache, Tomcat, and MySQL) with their corresponding event-driven asynchronous version (e.g., Nginx, XTomcat, and XMySQL) one-by-one. Our measurements with two application scenarios (virtual machine co-location and background monitoring interference) show that replacing a subset of asynchronous servers will shift the CTQO, without significant improvements in long-tail response time. Only when all the servers become asynchronous the CTQO is resolved. In synchronous n-tier applications, long-tail response times resulting from CTQO arise at utilization as low as 43%. On the other hand, the completely asynchronous n-tier system can disrupt CTQO and remove the long tail latency at utilization as high as 83%.

Original languageEnglish (US)
Article number36
JournalACM Transactions on Internet Technology
Issue number3
StatePublished - Oct 2019
Externally publishedYes


  • Asynchronous
  • Cloud computing
  • n-tier systems
  • Performance
  • Scalability

ASJC Scopus subject areas

  • Computer Networks and Communications


Dive into the research topics of 'Mitigating tail response time of n-tier applications: The impact of asynchronous invocations'. Together they form a unique fingerprint.

Cite this