Solaris LDOM – Another Solaris Virtualization layer I should be using
In a previous post I discussed Solaris Containers/Zones and why they were a good idea. Solaris has another layer of virtualization called Logical Domains or LDOMS. Oracle has rebranded it to “Oracle VM Server for SPARC” but it in most causes it is easier to call them LDOMs. Enough about nomenclature though. What exactly are these and why would you need another layer of virtualization?
Solaris LDOMs more closely resemble virtualization in the manner that VMware provides it. You have unique and fully segmented containers. These can run completely different operating systems or versions of Solaris. If you remember from the Solaris Zones article, the Non Global Zones (NGZ) share the same kernel of the Global Zone(GZ) hosting it. You can run an older version of an NGZ on a GZ but this is done through a compatibility library emulating it. The LDOM lets you have a unique instance.
LDOM does require that the processor support virtualization. For SPARC, this is primarily the processors based on the sunv4 architecture. More easily to remember/identify, Sparc T processors, whether its T1-T7 although there are other processors that support this. For x86/x64. Solaris did, in recent versions, start supporting this technology for x86/x64 as well but we will focus on SPARC processors for this article.
At this point, you are probably asking yourself, this is great but why do we need multiple layers of virtualization? LDOMs are great if you need a truly isolated environment. Perhaps you have specific versions of Solaris you need for specific purposes? For example if you have a production stack that requires Solaris 11.1 for the database and Solaris 10 for the App, you could easily set up an LDOM guest domain for each so that you could run those specific versions. Perhaps your app is 5-6 different applications that need some layer of segmentation because they cannot co-exist in the same OS instance. You could set up each one in a separate zone to achieve this.
As depicted above, another use case is for migrations. When it is time to deprecate legacy hardware but you still need the older Solaris versions because your app just will not run on a newer version or perhaps it would be unsupported/uncertified on the newer version and you do not want to deal with that scenario. Spinning up LDOMs and Zones are an easy and lightweight way to achieve this since processing power and RAM are usually in excess.
To achieve this, there are 5 main roles to an LDOM. Control Domain, Service Domain, I/O Domain, Root Domain and Guest Domain. The Control Domain is responsible for management of the LDOMs hosted on the physical server. It is usually combined with the Service Domain which is responsible for presenting certain resources to the Guest Domain such as disks. The Guest Domain is the actual Virtual Server Guest Operating System running. That leaves the Root domain and I/O domain. These two are usually combined onto the Control Domain as well. They are responsible for providing access to the PCI/PCIe busses. It is important to note that the Guest Domain is typically the only place you want to install your business applications to properly segment the environment.
Much like other Hypervisors, LDOMs can be live migrated from one physical server to another as long as they have shared storage and each server can see the same storage devices. This can be extremely helpful if you have an LDOM that is chewing through resources and you need to balance. When setting up the LDOMs you also set up any limits you would like for each of them in terms of RAM, CPU, etc.
Some of the main drivers for using both of these layers of virtualization come to processing power and RAM capacity outpacing the actual application needs. For example, I was involved in an extensive Solaris Datacenter migration where they were able to replace 30 racks of SPARC servers, SANs and network switches down to 6 racks of equipment and 5 total SPARC servers. With these 5 SPARC servers, a few hundred zones are being hosted through a dozen LDOMs. Management is much more simple because there are only 5 physical servers to manage. If a zone or LDOM needs to be bounced, zoneadm or ldm commands are easily within reach without having to send someone to the datacenter floor or remember the ILOM connectivity details.
Administration of LDOMs can be associated with Role-Based Access Control (RBAC). Perhaps you want to give certain administrators access to modify LDOMs but lower tier administrators to the Guest Domains/Zones. Easily done and important so that you can limit who has access to cause widespread configuration changes.
