Soft Resource Allocation in N-Tier Application Scalability
This research paper explores the impact of soft resource allocation on n-tier application scalability, discussing topics such as over-allocation, under-allocation, bottleneck issues, and proposing a practical algorithm for effective resource allocation. It also covers the performance implications in cloud computing environments and the experimental setup using the RUBBoS benchmark for evaluating different workloads.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
The Impact of Soft Resource Allocation on n-tier Application Scalability Qingyang Wang, Simon Malkowski, Yasuhiko Kanemasa, Deepal Jayasinghe, Pengcheng Xiong, Motoyuki Kawaba, Lilian Harada, Calton Pu
Outline Background & Motivation Background Motivational experiment Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation Solution A practical algorithm for good soft resource allocation Conclusion & Future Works 2 2 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Cloud Computing Environment Good performance + Cost efficiency High throughput + low response time High resource utilization Scaling applications on demand Bottleneck Bottleneck 3 3 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Soft Resource Allocation Soft resources in n-tier systems Threads, database connections, TCP connections, locks, etc. Is it okay to duplicate the same configuration of soft resource allocation? Bottleneck Thread pool Thread pool Connection pool Thread pool Thread pool 4 4 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Outline Background & Motivation Background Motivational experiment Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation Solution A practical algorithm for good soft resource allocation Conclusion & Future Works 5 5 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Experimental Environment (1) RUBBoS benchmark Bulletin board system like Slashdot (www.slashdot.org) Typical 3-tier or 4-tier architecture Two types of workload Browsing only Read/Write mix 24 web interactions C-JDBC Middleware for database scale-out Read: act as a load-balancer Write: send a request to all databases to keep consistency 6 6 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Experimental Environment (2) Emulab (http://www.emulab.net) Relatively modest testbed originally for network research Virtual network & physical machines (not VM) Hardware Server type Processor Memory Network Disk Specifications PC3000 in Emulab Xeon 3GHz 64bit 2GB 1Gbps 2 x 146GB 10,000rpm 7 7 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Experimental Environment (3) Software setups Function Web server Application server DB clustering middleware Database server Java Operating system System Monitor Software Apache 2.0.54 Apache Tomcat 5.5.17 C-JDBC 2.0.2 MySQL 5.0.51a Sun jdk1.6.0_14 Redhat FC4 Sysstat 7.0.2 8 8 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Experimental Environment (4) Notation 1 / 3 / 1 / 2 configuration 400-6-200 9 9 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Result of Motivational Experiment 1050 1/4/1/4 1000 400-6-6 Throughput [Reqs/s] 950 900 400-150-60 850 1/2/1/2 800 400-150-60 750 400-6-6 700 650 600 5000 5400 5800 Workload [# Users] 6200 6600 7000 7400 7800 10 10 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Challenge How to choose a reasonable soft resource allocation to match the hardware configuration? Good Soft resource allocation 1 Thread pool, DB connection pool performance Hardware configuration 1 Bad performance Scale out Soft resource allocation 2 Hardware configuration 2 Thread pool, DB connection pool Good performance Hardware configuration Soft resource allocation 11 11 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Focus of This Paper Evaluate two important soft resources Threads Database connections Show their performance impact on n-tier applications Over-allocation & under-allocation cases Special case of under-allocation Introduce a practical way to choose a reasonable allocation of soft resources 12 12 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Outline Background & Motivation Background Motivational experiment Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation Solution A practical algorithm for good soft resource allocation Conclusion & Future Works 13 13 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Performance Loss due to Soft Resource Over-allocation (1) 400-200-6 Sensitivity analysis: change DB Connection pool size in Tomcat 14 14 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Performance Loss due to Soft Resource Over-allocation (2) Goodput 1/4/1/4 CPU utilization in CJDBC 200 6 degradation Performance 6 50 Throughput with response time boundary 100 200 High allocation of DB connections in Tomcat degrades the system performance 15 15 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
JVM Garbage Collection Costs JVM Garbage Collection in CJDBC 200 8% more time used for GC over experimental time 6 Over-allocation of soft resources causes waste of critical hardware resources 16 16 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Outline Background & Motivation Background Motivational experiment Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation Solution A practical algorithm for good soft resource allocation Conclusion & Future Works 17 17 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Performance Loss due to Soft Resource Under-allocation (1) 400-6-200 Sensitivity analysis: change thread pool size in Tomcat 18 18 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Performance Loss due to Soft Resource Under-allocation (2) Goodput 1/2/1/2 CPU utilization of Tomcat 20 20 200 10 6 6 Under-allocation of soft resources causes inefficient utilization of hardware resources 19 19 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Outline Background & Motivation Background Motivational experiment Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation Solution A practical algorithm for good soft resource allocation Conclusion & Future Works 20 20 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Special Case of Soft Resource Under- allocation 30-6-100 Sensitivity analysis: change thread pool size in Apache 21 21 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Performance Loss due to Under- allocation of Apache Threads Goodput 1/4/1/4 CPU utilization in CJDBC 400-6-100 400-6-100 100-6-100 50-6-100 30-6-100 30-6-100 Low allocation of Apache threads degrades the system performance 22 22 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (1) 30-6-100 30 > 6 * 4 Why are 30 threads in Apache not enough? 23 23 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (2) 1: HTTP request 2 4: HTTP response 3 5: FIN reply Waiting for TCP FIN reply from the client 24 24 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Non-Trivial Correlation between Apache and Tomcat Threads (3) 1: HTTP request 2 Communicating with Tomcat 4: HTTP response 3 Apache thread active period Waiting for TCP FIN reply from client 5: FIN reply The long waiting time for FIN reply from clients is unpredictable and frequently happens under high workload 25 25 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Concurrency of Apache Threads Concurrency for 30-6-100 Concurrency for 400-6-100 Connecting to Tomcat Large number of soft resources in front tier acts as a buffer providing stable workload for lower tiers 26 26 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Summary of Experiments 1. Over-allocation of soft resources causes waste of critical hardware resources 2. Under-allocation of soft resources causes inefficient utilization of hardware resources 3. Large number of soft resources in front tier acts as a buffer providing stable workload for downstream tiers 27 27 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Outline Background & Motivation Background Motivational experiment Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation Solution A practical algorithm for good soft resource allocation Conclusion & Future Works 28 28 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Soft Resource Allocation Algorithm Key idea: allocating soft resources globally to utilize the critical hardware resource as efficiently as possible 29 29 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Soft Resource Allocation Algorithm (1) 1. Identifying the critical hardware resource first Throughput vs. Workload (1) Bottleneck server 1050 Throughput 900 750 600 5000 5800 6600 7400 Workload 30 30 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Soft Resource Allocation Algorithm (2) 1. Identifying the critical hardware resource first 2. Allocating proper soft resources for the bottleneck server Throughput vs. Workload (1) Bottleneck server Throughput knee 1050 Throughput 900 Minimum Saturation workload 750 (2) Ave. number of active threads 600 5000 5800 6600 7400 Workload 31 31 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Soft Resource Allocation Algorithm (3) 1. Identifying the critical hardware resource first 2. Allocating proper soft resources for the bottleneck server 3. Allocating proper amount soft resources in other tiers (3) Dependency between current tier and bottleneck tier (1) Bottleneck server = = * L TP TP RTT Re / TP q apache cjdbc ratio (2) Ave. number of active threads = ( * / Re ) L cjdbc L RTT q apache ratio ratio 32 32 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Outline Background & Motivation Background Motivational experiment Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation Solution A practical algorithm for good soft resource allocation Conclusion & Future Works 33 33 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Conclusion Achieving good performance by scaling n-tier applications in Cloud requires a unified exploration of both hardware and software Contributions: We showed allocation of soft resources has a big impact on the total performance of an N-tier system We showed to decide a proper soft-resources allocation is a complex problem, especially in cloud environments which requires dynamic scaling-out/in We gave a practical algorithm for proper soft resource allocation 34 34 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Future Work Explore more soft resources such as locks, buffer/queue Explore more efficient ways to find proper soft resource allocation Explore the impact of soft resource allocation in virtualized environment 35 35 25th IEEE International Parallel & Distributed Processing Symposium 19 May 2011
Thank You. Any Questions? Qingyang Wang qywang@cc.gatech.edu
