Memory Resource Management in VMware ESX Server
Explore the memory resource management techniques in VMware ESX Server, including ballooning, idle memory tax, content-based page sharing, and more. Learn how server virtualization optimizes hardware resources and improves performance.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Memory Resource Management in VMware ESX Server Carl A. Waldspurger Vmware Inc. Presented by Wesley Coomber wcoomberEECS 582 W16 1
Agenda Motivation for VMware ESX Server Ballooning Idle Memory Tax Content-based Page Sharing Hot I/O Page Remapping Discussion EECS 582 W16 2
Motivations Why Server Virtualization? Individual servers are often underutilized, so it makes sense to consolidate them as VMs on a single physical server to simplify management and reduce costs. Present Industry trends Server consolidation and lots of cheap shared-memory multiprocessors-> Server Virtualization has room for improvement Smarter multiplexing of physical resources EECS 582 W16 3
What is VMware ESX Server? A thin software layer that multiplexes various hardware resources efficiently among a number of virtual machines How is this different? Traditional virtual machine systems had a hypervisor that runs on top of a typical operating system and intercepts I/O device calls from VMs to handle them as a host OS system call ESX Server runs on top of the system hardware directly, which provides better I/O performance and more control over resource management. Can run multiple operating systems with any necessary OS modification EECS 582 W16 4
What is VMware ESX Server? (2) ESX adds another layer of abstraction by virtualizing real physical memory (machine address) into a physical address software abstraction that makes the VM think it has access to hardware memory ESX Server has a pmap for each VM Also separate shadow page tables for the processor Server can transparently remap physical pages and monitor/manipulate guest memory accesses Over-Commitment of Memory! EECS 582 W16 5
Ballooning A technique to implicitly reclaim memory How is this different? Traditional approach is to have another level of paging, and then moving a VM s physical pages to disk. This is bad because it necessitates a meta-level page replacement policy. (Eg. Which VM do I take memory from AND which pages to reclaim?) When ballooning is impossible or too slow, then the system defaults to random page replacement. http://spongebob.wikia.com/wiki/Life_of_Crime EECS 582 W16 6
Ballooning (2) Balloon drivers poll the server once a second EECS 582 W16 7
Ballooning Benchmarks Dbench benefits a lot from extra memory EECS 582 W16 8
How memory management works in ESX Reminder: ESX server gives each guestOS the illusion of a physical address space that starts at 0, when each address is actually mapped to non-contiguous actual hardware machine address 3 variables go into allocation of memory to each VM. Min (guaranteed) size Max size Memory shares http://clipartix.com/reminder-clip-art-image-12468/ EECS 582 W16 9
Idle Memory Tax A technique to tradeoff some performance isolation for efficient memory utilization How is this different? Traditional approach is to have a pure proportional-share algorithm that maintains specific ratios of memory between VMs This is bad because it lets idle clients hoard memory and waste it while working clients with meager shares are under severe memory famine. https://commons.wikimedia.org/wiki/File:Logo_of_the_Intern al_Revenue_Service.svg EECS 582 W16 10
Idle Memory Tax (2) Charge a client more for its idle pages than for the ones it is actually using When the system needs memory, pages will be claimed first from clients that are not actively using their full allocated memory. Tax rate = max fraction of idle pages that can be taken from a client (defaults to 75%) ESX Server measures idle memory by statistically sampling the VM working sets to generate estimates of the amount of actively used memory for each VM. EECS 582 W16 11
Idle Memory Tax Benchmarks Two VMs with identical share allocations of 256 MB in an overcommitted system. EECS 582 W16 12
Content-based Page Sharing A technique to safely share memory between virtual machines How is this different? Traditional approach is to identify redundant copies of pages, delete them, and then map the physical pages to the single original CoW copy. This (traditional way) is bad because it requires several modifications to the guest OS for it to work. http://hamster.wikia.com/shareCarrot.jpg EECS 582 W16 13
Content-based Page Sharing (2) Content-based page sharing completely removes concerns for sharing away from the guest OS. (no need for modifications) Need to scan for sharing opportunities! ESX server hashes the contents of candidate pages and the hash is indexed into a table of other scanned pages. If there is a match with a hint frame, then a full comparison is done. If the pages are indeed identical, then the server maps each physical page to a single machine page and marks it CoW EECS 582 W16 14
Content-based Page Sharing Benchmarks Identical linux VMs running SPEC95 benchmarks EECS 582 W16 15
Hot I/O Page Remapping Modern processors can address up to 64GB of memory. However, many devices that use DMA for IO transfers can only address up to 4 GB of memory. Traditional approach-> copy high (>4) memory into a temp buffer in low memory. This is expensive, and even worse in the case of VMs since VMs that think they have low memory might actually be mapped to high memory! ESX server tracks hot pages that are involved in a lot of I/O, and when it reaches a certain threshold, the page is transparently remapped to low memory. EECS 582 W16 16
Discussion Is the hot page remapping feature (published 2002) still useful for modern virtualized servers? What can be done to help alleviate the limitations of ballooning (balloon driver can be disabled or unavailable while guest OS is booting)? Is there a better page replacement policy for this system compared to the implemented randomized page replacement policy? wcoomberEECS 582 W16 17
