By Anil Desai

published: Friday, February 22 2008

This article is the first of two parts on managing virtualization performance:  In this initial installment, I’ll focus on why and how you should monitor your host servers and virtual machines.  The goal is to get more insight into your virtual infrastructure (before someone less compassionate does it for you).

Benefits of Performance Management

I can only imagine that the view from the bridge of a Star Destroyer can be breathtaking (yes, the pun was intended).  But it can’t be easy to manage an entire fleet of starships bent upon galactic domination just by looking at a bunch of flashing lights.  Similarly, it’s hard enough o gauge physical server performance through the use of visual metrics.  Add in virtualization, and you have a new layer of complexity to consider.

Let’s face it, though – like just about anything worth doing in it, managing performance takes time, knowledge, and effort.  Those things tend to be in short supply.  So before we dive down into the details, let’s look at some of the benefits of managing performance.  Figure 1 provides an overview of the some of the big ones.

Figure 1: Potential benefits of implementing performance monitoring

To sum it up, monitoring the environment allows you to make better decisions.  The business side of the organization benefits by being able to plan for new deployments and to create appropriate budgets.  Some of the other benefits are just standard IT best practices, such as being proactive.  If you’re looking for benefits from server consolidation, for example, you can better quantify your success (or lack thereof) by monitoring performance.

The “alternatives” really aren’t all that attractive.  One option is remaining in the reactive role that is so common to IT environments.  In that case, you would simply “fight fires” as they occur.  Is a user having a problem with critical transactions timing out?  Well, it’s time to scramble to reallocate the VM (often involving downtime and people wishing to use their Dark Force Powers on you).  Another option, which is just as glamorous, is to not monitor things at all.  After all, users will call you when there’s a problem, right?  Right?  Beware that you might start to feel like you’re stuck in a garbage chute with the walls closing in on you.  If you like those approaches, there’s no need to read any further.  But for the rest of us…

Monitoring Host Servers

Virtualization adds a level of abstraction between individual VMs and the hardware resources that they are ultimate using.  It’s the role of a Hypervisor or virtualization management layer to coordinate requests.   The first approach to monitoring performance starts at the host server.  By watching resource utilization in critical areas – CPU, memory, storage, and network – you can get an aggregate view of the performance of the server.  Monitoring virtualization hosts can help you answer the following questions:

• Which servers are underutilized and which have additional capacity?
• How much remaining capacity do I have on my production systems for new deployments?
• Which servers are the best candidates for new VM deployments?
• Which physical machines would benefit most from hardware upgrades?
  

These are all important questions, and host monitoring can provide valuable insight.  But it leaves some important stones unturned…

Monitoring Virtual Machines

By monitoring physical resource utilization at the host level, you can get a good high-level view of overall server resource consumption.  But when there’s a problem – perhaps CPU spikes that last for hours – what’s the problem?  Is it one VM that’s monopolizing the system?  Or, is it just a combination of VM usage patterns that are the culprits?  Monitoring at the VM level will help answer these questions quickly and easily. 

Some virtualization platforms will allow you to monitor basic statistics such as CPU and memory utilization per VM at the host level.  If you want more in-depth information, however, it can be more difficult.  For example, if you want to monitor which application or service within a guest OS is consuming the most resources, you’ll need to start collecting statistics from within each VM.  The details, of course, will vary based on the guest OS and its workload.

Statistics to Monitor

So far, the discussion has been at a pretty high level.  You know how you can implement monitoring, but information should you collect.  Let’s zoom in on what you might want to monitor.  The statistics shown in Figure 2 are a good start for monitoring at the host and guest OS levels. 

Figure 2: Common performance statistics to monitor

Most operating systems and virtualization platforms will provide you with some way to monitor overall performance.  Unfortunately, when you’re doing this manually, you’ll need to use different techniques for different platforms.  For example, all current versions of the Windows platform include the modest but powerful Performance Monitor – an excellent tool for both short- and long-term monitoring.  The challenges, however, are primarily related to monitoring large numbers of physical systems (thanks to server sprawl) and large numbers of VMs (thanks to VM sprawl).

Monitoring Resource Usage Patterns

One of the biggest advantages of virtualization technology is its ability to decouple workloads (such as specific applications and services) for the underlying hardware.  No longer do you need to allocate a full physical server for that Accounting app that one user needs at the end of every month.  However, with the power to easily move workloads around comes responsibility.  Usage patterns can change over time, and decisions you made during initial deployment of a VM might no longer be valid.  This is where monitoring resource usage patterns can be helpful.  Figure 3 provides a simple example.

Figure 3: Tracking CPU utilization over time for various servers

This data can be useful for several purposes.  First, you can establish an overall baseline for standard usage patterns of your VMs.  When numbers start to deviate significantly, it’s probably time to closely monitor the source of the change.  Second, you can use the data to make better decisions about where VMs should be deployed.  That’s a little foreshadowing for my next article.

The Saga Continues…

By properly monitoring virtualization performance, perhaps you can avoid the shortness of breath symptoms that are so common when working for employers like Darth Vader.  With a little bit of planning and effort, you can get a tremendous amount of insight into how your VMs and their hosts are running.