By Mike Wookey

published: Tuesday, April 22 2008

MW: From the beginning of Sun, since we really came up with a target network for the computer, we’ve all been all about trying to make best use of the resources on the network. And if you really cast your virtualization net-wide, you will realize that virtualization is all about making the best use of the resources available to bring them to the applications or the needs of the user. So if we cast back, we look at NFS, our Network File System invented in ’85 which really spurred some technology toward this whole network computing model. And then we went through a series of differentiating technologies which were, in many ways, evolutions of the IT market at the time.

Anything from resource management to application sense, through to our more recent inventions like chip multi-threading inside a Niagra chip platform. And then, as we move forward more into what is known as compute virtualization today represented by Xen and the end-worker, the Hypervisor technologies, we’re now looking at how we can take some of our expertise and our learnings over those years and apply them into the market of consolidation and resource management at the compute model.

When we look at this, we look at how we leverage and how we use those technologies over the years; how we actually share resources on the network; how we share resources within the system itself; how we built schedulers in the past to run inside of Solaris. Back in the late ‘90s, Solaris’ operating system was important because of its ability to scale, because of its ability to manage resources efficiently, and run applications such as Web servers during the big dot com years extremely efficiently and share them. All of this maturation of technology that we have, all of these components and experience can be brought to bear, we believe, to produce our next release, our next focus on virtualization: the project that’s called xVM today.

MW: We’ve been observing Xen for a while. Again, speaking about all the technologies, to some extent we’re famous for our SPARC processor technology, and as we’ve invested more into the X64 and X86 base from a systems perspective, and indeed we released solaris on X564 and X86. We’ve been looking for a hypervisor virtualization technology to enable us to effectively optimize rescources on that particular platform. As we looked around and looked at the of capabilities, obviously one of your choices is always to build your own, but that is a significant task that requires many years of investment and resources being tied up. And out there in the Open Source is Xen, and Sun has been investing in Open Source for many years now and we've really apmed up our investments in Open Source through many other avenues such as Open Office and through over sourcing our operating system core. So Xen is influential in the Open Source and is actually a mature product, which was extremely important to us to pull something that has already reached a certain level of maturation in the marketplace to other vendors’ releases, and was mature to the point where it wasn’t just offering pure power virtualization with modified operating systems, but can also run Windows hardware and HVM support from info and AMD.

The amount of investment that is going into Xen from the community – and this is a wide community, we held the Xen community event at our Santa Clara campus and it was interesting that out of 200 attendees, which is pretty large for an Open Source project with a focus such as Xen, it was really encouraging to see how many contributors, given the fact that Xen came from the university network, the academia, and now to see how many Open Source contributions are coming in from community members such as IBM, Cicso and Red Hat and all those members. They are inputting important technologies that will effectively lift Xen’s capabilities to new heights. We really wanted to take advantage of that and really work with the community to put some of our knowledge and capabilities that we’ve built up over the years and base our product line, our supported product on this solid piece of technology that is evolving quickly and help evolve it in the future. A lot of Sun’s direction now is toward this community involvement the to just be able to attach it to a community that exists already and to be able to grow that community with our own product line by involving our own Open Source and open-sourcing everything we do, as well as giving back to the community is extremely important to us.

MW: One of the challenges that exists in the marketplace today is that technology, as we all know, is moving forward extremely quickly. It’s moving forward even quicker in the virtualization space. Many users use virtualization technologies to a limited level, and they use it to a level that they can either understand or can adopt within their own structures in their datacenters.

From our conversations with many of our existing customers, many of the issues seem to be around the management of the virtualization. They can only use the power that the virtualization engine, such as Xen, exposes. If they can actually operate in, not only in a single manner, but at scale. They want to be able to deploy up to thousands of systems and want to be able to virtualize tens of thousands of datasystems in some cases, but they need to have confidence in their management. They need to be able to configure, they need to be able to reconfigure, and take advantage of all of the new capabilities that virtualization brings, such as the ability to execute disaster recovery, and better resource management through load balancing and load analysis.

To do this, they can’t just be expected to go to command lines and issue commands on single systems. They can’t expect to operate these complex technology components through single, simple management sources. What is really needed is this combination of management and virtualization, where the two go in lock step moving forward step and involve in lock step. So the xVM strategy and the xVM here stands for the interception of virtualization and management. It is really to create a product line that allows us to move forward in lock-step on the one hand, offering management tools which allows you to expose the capabilities of the virtualization, on the other hand to bring forth new virtualization capabilities as we evolve the community and as we help the community evolve them moving forward.

One of the challenges that we face here is that the management and automation market is extremely complex. There are many vendors out there who offer management with varying degrees of virtualization and they offer varying degrees of guest management. So the approach that we’re taking is to really offer holistic management. xVM to us is the virtualization capabilities, the management of those virtualization capabilities, and the management of your greater environment that actually supports the virtualization capabilities themselves. The final piece of the puzzle is really to Open Source everything that you do. It’s to create, it’s to foster a community around the virtualization pieces, which is today, in this particular instance, mostly represented by Xen, and the management components which we will be open sourcing through our OpenxVM.org, and pulling the community together and fostering it and basically developing all of this transpowerment to the community, and giving back everything we do. All of those components create a solid strategy moving forward.

Moving on, as we look forward though, it is important to realize that, to us, virtualization goes beyond just Xen or the Xen technology. It actually is looking at things like how to create virtualization technology on top of our smart chips. We’ll be leveraging our Logical Domains project and , again, the management interface will appear to be exactly the same as it is for Xen, but instead of virtualizing it at 64 or 36, you’re virtualizing SPARC. Then we also have other virtualization technologies we want to bring in, things like our containers capability in Solaris, which is, effectively, OS level virtualization. We want to be able to manage that and essentially give the user the choice, based on the chip architecture and the system type they want to use, based on the particular tier of virtualization they want, yet still maintain the integrity of the management interface and make sure that everything appears the same to them so that they only have to learn how to manage something once, yet they can choose and vary that deployment criteria.

MW: The xVM Ops Center represents a number of years of intellectual property build out that we have been doing in the management space. In many respects we have been working on and focused on this for about a year and a half, but on the actual technology for two years prior to that. It is the synthesis of data center automation capabilities from Sun. It is this particular product that we are launching, which will provide the full virtualization management capabilities that I spoke about before. In it's warmed-up zero variant, which we will be releasing extremely shortly in February, it is about managing, at least initially, the physical aspects of your data center, so the actual hardware systems themselves, the operating system and the application stack that goes above that, but it’s not traditional management of these levels, it is true data center automation that is really its focus. Data center automation, for example, it’s about that, so we can discover, we can manage the hardware itself without any operating systems being on it, which is really important when we want to talk about virtualization because if you install a hypervisor on a piece of hardware on to bare metal then there is no OS – there’s just whatever the hypervisor at the domain zero provides you on the control end. So we can sort directly to the hardware using standards like IPMI and SNMP and then have a great adaptive understanding of it, and we can update and manage the firmware levels. All of this seems terribly mundane compared to virtualization and migration, however, it is essential for those things to work and we can’t lose focus of that.

Then we can run compliance reports and make sure all the firmware is at the right levels so the hardware is operating as you would expect. And moving up the stack, we can look at provisioning – we can provision out software on to the bare metal itself, en masse, thousands in some cases, depending on your network configuration and bandwidth. Included in that, of course, as we release the 2.0 release of xVM Ops Center which we’ll release at the same time as the xVM Server, the X64. We will be able to network and store out our xVM Server appliance out to the bare metal on that. The most important piece to apply to both virtual machines and normal OS’s running on top of bare metal is the lifecycle managements of the operating system and the application stack itself. At the core of this engine is the concept of compliance reporting, ensuring that you’re running the software you expect, the versions you expect, for any host of reasons, such as security – making sure you’re running all the latest security updates- ensuring all of your tier-1 Web farm is running your identical software, and you’re not having version drift which can often lead to all sorts of problems showing up, making sure you can see the analysis of what will happen if new updates are installed before you actually update them – we have concepts of simulation. This whole life cycle component is proving to be invaluable to the customers we’ve already shown the product to and they have used it in a test because it really does allow them, in high scale, to be able to see problems and then be able to reconcile those problems, and understand the concept even without virtualization of snapshotting and recovery and restore across hundreds and thousands of instances.

And then if we think about one of the challenges from the management perspective that virtualization brings in, it’s scale. For every operating system that used to exist on and X64 box, there may now be 10, 20 or 30, depending on how your utilization model is. All that does is really make the management problem significantly harder. On top of that, it makes the security model harder because of the roles and the access controls that the management team needs to have needs to be extremely strong. For example, we’ve already rolled out a pre-release of Ops Center 1.0 into Texas super computer, the largest super computer and high-performance grid currently being built in the world and Ops Center will be managing its capability. So scale is important to op-center, depth, life cycle management, and then, as you move into our other op-center, again, with the release of the xVM Server, we are looking at open APIs so you can take the management product and you can interface it into the rest of your data center management capabilities. We are basing it around WX management standards and API. We are looking at, frankly as a concern, an AJAX interface so that you can access your management interface on the browser, anywhere, and the management tools themselves are designed just to fit in. They are multi-tiered as you wish from a deployment perspective, they use common protocols, they are fully inspectable, and they have the ability that the user, the administrator who really wants to see what the management system can do, they can just look at it.

MW: The xVM Server, you can think of it as a software appliance from Sun. As such, we are packaging up a lot of technology which is available from Sun today, together with a lot of new technology, and creating a micro-software appliance that can be deployed onto your system, your bare metal system, and it will provide full system virtualization capabilities. And built into this micro-appliance are, obviously, technology from the Xen community, the hypervisor itself, as well as a good percentage of the latest technology that we at Sun have been building and will be embedding into future versions of Solaris. For example, we have another product which is being spoken a lot about in the press called Indiana, it’s our open Solaris distribution, and in this distribution, we have implemented a new packaging system to allow capabilities to be easily managed from a software distribution standpoint. This actually allows us to shrink down, significantly, the size of the operating system. We have ported the technology from the Xen community into Solaris, and then we have shrunk down that domain-zero, that control domain significantly, using this new packaging capability. Into that, we have applied some pretty cool technology, and the first one is Crossbow. Crossbow is network virtualization. What this allows you to do is two important things to virtualization systems: Number one it allows you to assign virtual NICs and virtual MAC addresses and then apply bandwidth quotas to each virtual NIC. So it allows you to subdivide how much each virtual NIC and therefore, ultimately, each virtual guest, each virtual machine, how much bandwidth it gets.

The second thing is using a tagging mechanism to tag on virtual MAC addresses and this is a pretty important feature because in a simple virtualization scheme, where say you have three guests on a single server, and that single server has one physical network interface, even though there’s three guests on that single server, each one of those guests, once that traffic makes it onto the physical network, will be tagged with the same MAC address. So your conventional networking equipment will only see one MAC address and therefore will only have the ability to prioritize and route based on that one MAC address. So, to the network, those three virtual systems are only one system. What Crossbow allows you to do is expose MAC addresses for each of those three systems. So each of those three systems can now be routed and prioritized by your normal physical networking equipment as appropriate. That becomes really important when you are trying to virtualize datacenters or tiered applications with one box. That technology has been in development, inflated for Solaris inclusion and future versions of Solaris, but we are going to be providing that technology into the xVM Server earlier than it would be conventionally available in Solaris.

Other features include things like DTrace. DTrace is a technology that allows us key observability of execution paths within a control domain, it allows us to see potential problems and bottlenecks and resource problems. It allows us to flag problems that may affect the user and help understand these things so you’re exposing that to our management interface and FMA. Our Fault Management Architecture is built into the core of the xVM Server and this allows us to flag anything that ranges from potential hardware issues up to potential operating system internal issues and allows us to provide a very controlled and very succinct fault information up to the user. Again, this is very important when we talk about true production in a stable environment. ZFS – Zettabyte File System – we will be leveraging next for storage of guest information locally on the xVM Server, as well as internally inside the xVM Server for management of subsequent updates on the xVM Server itself and ZFS has the ability to snapshot and will use this to ensure that any updates or subsequent updates to the technology can be maintained and integrated.

Other key features we’re looking at, I mentioned the packaging system previously, CIFS support is being integrated into our Solaris code base and will be exported to the xVM Server so you can use both CIFS and NFS to attach network images of guests if you so wish into the xVM Server. On top of that, we’re leveraging all of the years of expertise and optimization that have gone into the core Solaris kernel around, things like multi-processor, multi-threading capabilities and the scheduling of processes and this actually give us significant performance capabilities because essentially what the control domain is doing is scheduling across guests and managing resources. Take all of that and wrap it inside a single xVM Server management interface, so that all these actions, all the capabilities, not just virtualization, but all of the domain zero management that needs to be done around network configuration, routing and all the usual mundane things, all are accessible and managed through a Web interface. Then, if you choose to use Ops Center, that interface will look exactly the same as Ops Center, and you can migrate seamlessly through the interface into the greater management world.

MW: So the key message, just to reiterate, is there is a lot of state-of-the-art technology going in here. xVM will be the first product to use much of this new technology from Sun. Much of this technology you’re not going to be seeing in true product terms until a later version of Solaris. So that is a key aspect for us: the ability to expose things like Crossbow early, the ability to expose some of the key features around ZFS and our new packaging scheme into a new production world into this mini-appliance. It really is, taking these many years of experience that we discussed earlier and being able to imbed that into something that is incredibly easy to use. With that incredibly easy-to-use mantra, we’re also given the management tools that maintain that easy-to-use scale. So this isn’t just going down roads with a single entity and you’re storing it on your bare metal, this is download it, then install it, then manage it on a larger scale. Then, on top of that, don’t just manage your virtualized environment, manage the physical aspects of the server, manage within the guests themselves making sure their operating systems application stacks and are compliant, manage all the patches and updates, manage across Windows, Solaris and Linux, using the same tools, and then provide performance solution to reduce the tax that virtualization puts on the hardware. So performance, scaleability and manageability is our strategy moving forward and, what we believe, is our key differentiator to the competition. Then, through all of this, we’re in the open, Open Source everything, provide all of our specifications online, work with the community to add in new features. Then, the final piece is, as we discussed, provide open APIs to people who want to integrate into it using existing standards such as WS-MAN and leveraging as standards from standards bodies such as the DMTF to provide support such as OVF virtual appliance descriptions, and some of the work they’re doing around the common information model and virtualization extensions for that will go into our API.

MW: Well, obviously, we’re really going to focus on the evolution of the xVM strategy. Next up is going to be the xVM Server for SPARC, which I discussed earlier, so effectively taking all of the great capabilities exposed inside the xVM Server X-86 and allowing them to expose to the SPARC hypervisorand the model on the applicationing of software will be very similar, and the management interface will be very similar. Moving past that, we are going to look at container management, the Solaris containers, how do we expose that subtly different style of virtualization, which has its own advantages – you don’t need to pack many applications and virtual containers inside a single operating system. How do we expose the magic? How do we make it look the same so that people who have become comfortable with managing the xVM X-64 can look at containers and say, ‘Okay, this is just a deployment decision.’ Then, as we move forward, as we accelerate this program aggressively, and as we bring in the community, we are going to start looking at some of the more autonomic behavior.

One of the areas which kind of mates and joins with another key strategy at Sun, that of eco-responsibility. How do we leverage the ability that virtualization gives us to provide true power optimization and conservation? This goes beyond just migrating guest domains from Machine A to Machine B and shutting down a machine that isn’t used, it goes into more advanced analysis that we are doing on the eco side of the world that we are doing to understand the behaviors and the power consumption of systems, given certain types of system load and how do you manage that and how do you impart logic on that so that a virtualization system can make intelligent decisions about power optimization and then provide that as a choice to the customer, ultimately, quality to the customer.

Last off, we’re obviously moving on and looking the more traditional world of disaster recovery and some of the problems that really impact our customers today, we’re looking at advanced utilization of load management as an extension to that eco-work we just mentioned, and the other area which is going to show up in xVM Server 1.0, but will definitely flesh-out, is really looking into the virtual appliance models. How do virtual appliances truly affect the distribution of software? How do they truly affect some of the problems affecting our customers around compliance, around stability, around changeability, and adaptability. That is kind of our focus moving forward and it’s widening as we get behind the core initial packaging and technology delivery, and we’re going to broaden pretty quickly, and our portfolio capability is going to enhance rapidly.

MW: The future is interesting. In many respects, virtualization frees us from many of the bounds that we had before. Traditionally, we have worked with the physical entity being the handcuffs that we’ve all worked from and software as being tied to physical entities. I see, as we look forward, the movement towards some of the high-performance compute models with small grids. In many respects, small grids of compute can free you completely from the hardware, and if you lay a virtualization layer along the small-compute grid, the mobility that is provided for applications and the ability to manage and to deploy applications and to bring them up and bring them down on an as-needed basis is enormous. I see the combination of a lot of the work that’s happening in HPC [high performance computing] today around effectively taking the use of resources and improving the use of resource utilization on the grid, coming together with a lot of the work around resource management and utilization in virtualization, I see these two things coming together pretty quickly. So I think what we’ll see is virtualization playing an extremely key component in compute farms and the likes of the large Web capabilities in deploying large grids which, relying on virtualization will allow them to deploy out new content and new software as needed. Then, based on user need, they will be resourced appropriately and dynamically leveraging that future capability of virtualization.

Related Links:

SunMicrosystems.com , Solaris , XVM , Xen.org