Gareth Rushgrove posted an article recently, called The End of the General Purpose Operating System, in which he explained how he sees the operating system becoming an implementation detail of the higher level software.

By general purpose OS I’m referring to what most people use for server workloads today – be it RHEL or variants like CentOS or Fedora, or Debian and derivatives like Ubuntu.

By end I don’t literally mean they go away or stop being useful. My hypothosis is that, slowly to begin with then more quickly, they cease to be the default we reach for when launching new services.

In all of these cases the operating system is an implementation detail of the higher level software. It’s not intended to be directly managed, or at least managed to the same degree as the general purpose OS you’re running today.

And I agree with Gareth, but with a different long-term end. He alludes to this in his blog post, specifically mentioning the diffuse distinction between host process supervisor (systemd) and the container process supervisor (Docker engine), which reminded me of that whole Unikernel thing.

I’m a pretty solid believer in the Unikernel (and Library OS) developments, and I think they are a great innovation in the OS-space. They’ll at least fuel the efforts of making more efficient, very specialised Operating systems that will replace the current general purpose operating system in the datacenter.

Remind me, what’s a Unikernel again?

Basically, a unikernel is whatever you need (your application, any middleware and application dependencies, OS libraries plus kernel) in it’s purest form: everything else is stripped (even debugging tools). It has everything it needs to run, including a bare set of device drivers (usually to run on a hypervisor like Xen or KVM). In comparison to traditional Operating Systems, a unikernel only has a single address space, without a distinction between kernel and user space code. Of course, this is really only suitable for application developers who have figured out how to separate application functions into a micro-service architecture. The advantages of Unikernels are pretty promising, though.

Library operating system is a build system which links a group of libraries representing traditional OS functions (networking, storage, timekeeping, middleware) with an application to produce a unikernel. The Library OS is the required core concept behind a Unikernel, since you can’t just make a unikernel out of any ol’ Operating System.
In some ways, Microsoft has been working towards Library Operating Systems with Windows Server Core (2012) and Windows Nano Server (2016). I imagine Microsoft actually developing a Library OS soon.

What does this have to do with Containers?

If you zoom out a little, and look at why Containers are pretty awesome in comparison to server virtualization, you see the benefit really is in the Operating System: instead of running an Operating System per Virtual Machine, you get to run a single shared Operating System.

Except, this is not entirely true in most environments: each container has it’s own Operating System base layer. Just look at documentation here, here and here. Yes, that is a full (albeit optimised) Operating System filesystem for each container.

Isn’t that weird? One of the most advertised advantages of running applications in containers, and it’s only partly true: base images are in fact smaller than a full installation in a VM (~200MB vs. ~2GB for Ubuntu), and there’s some layering and sharing advantages.

And this is where Unikernels and Library Operating Systems come into play. Instead of sharing a ‘fat’ kernel and Operating System, separate code artifacts (functions or services in the micro-service architecture) in lean unikernels running on a hypervisor or container host.

Unikernels as the Unit of Software

While I agree with Gareth that containers are becoming ‘the unit of software’ for both distribution and deployment, I see room for improvement in the mechanics behind the container format. Simply put: Unikernels can help bring down the amount of data in each ‘docker pull’ for a given application.

And this is where general purpose operating systems and container-optimized versions of those operating systems, coming to an end. I bet that the OS will become an implementation detail at build phase, not a base layer at the ship phase. Heck, even container hosts will be highly specific versions of an Unikernel.

Will the Unikernel really rise?

I don’t know, but I’m a fan of the concept, and I would love to chat to you about this with others that have any experience. It is, after all, an emerging technology with yet unknown implications. I’d be very curious to see which direction tech startups like Docker are taking this, and how enterprises are considering the concepts.