Improving the state of art in container cloning

Since early August I’ve been looking at improving the state of the art in container cloning with an ultimate goal of making Juju faster when used in a local provider context, which happens to be where I spend most of my days lately. For those who don’t know, the local provider in Juju uses LXC with the LXC-provided ‘ubuntu-cloud’ template in order to provide an environment that’s as similar as possible to provisioning a cloud instance elsewhere.

In looking at the various storage backends supported by LXC and experimenting with each of them, I’ve stumbled on various issues, from broken inotify support in overlayfs to random timing issues deleting btrfs snapshots. Eventually I’ve discovered the holy grail of LVM Thin Provisioning and started working on a patch to LXC which would allow it to create volumes in a thin provisioned pool. In the meantime, Docker announced their intent of adding support for LVM Thin Provisioning too. I’m happy to announce that the work I started in August is now merged into LXC as various separate pull requests (#67#70#72#74) and is already available in the LXC Daily PPA. I’ve even created a Gist showing how to use it.

As pointed out by a colleague today, Thin Provisioning support should soon land in Docker itself. I applaud the initiative, and am really looking forward to seeing this feature land in Docker. I wish though there was a little more coordination going on to get that work upstreamed into LXC instead, to the benefit of everyone. Regardless, I’m committed to make sure that the differences between whatever ends up landing in Docker and what I’ve added to LXC eventually converge. One such difference is that I’ve simply shelled out to the ‘lvs’ command while Alexander is using libdevmapper directly, something I’ve looked at doing but felt a little over my head. I’ve also haven’t got around to making the filesystem default to ext4 with DISCARD support yet, but that’s on the top of my list.

So without much ado, let’s look at an example with a loopback-mounted device backed by a sparse-allocated file.

$ sudo fallocate -l 6G /tmp/lxc
$ sudo losetup -f /tmp/lxc
$ sudo pvcreate /dev/loop0
$ sudo vgcreate lxc /dev/loop0
Nothing special so far, simply a 6G sparse file, mounted loopback and then a VG named ‘lxc’ is created within it. Now the next step, creating the Thin Provisioning Pool. The size of the LV cannot be specified as 100%FREE because some space needs to be left for the LV metadata:
$ sudo lvcreate -l 95%FREE --type thin-pool --thinpool lxc lxc
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
lxc lxc twi-a-tz- 5.70g 0.00
Now if you have a recent enough LXC as the one from the PPA above, creating a LVM-backed LXC container should default to creating a Thin Provisioned volume on the thinpool named ‘lxc’, so the command is pretty simple:
$ sudo lxc-create -n precise-template -B lvm --fssize 8G -t ubuntu-cloud -- -r precise
$ sudo lvs
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
lxc lxc twi-a-tz- 5.70g 17.38
precise-template lxc Vwi-a-tz- 7.81g lxc 12.67
If you wanted to use a custom named thin pool, that’s also possible by specifying the ‘–thinpool’ command line argument.
Now, how fast is it to create a clone of this container? Let’s see.
$ time sudo lxc-clone -s precise-template precise-copy
real 0m0.276s
user 0m0.021s
sys 0m0.085s

Plenty fast I say. What does the newly created volume looks like?

$ sudo lvs
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
lxc lxc twi-a-tz- 5.70g 17.44
precise-copy lxc Vwi-a-tz- 7.81g lxc precise-template 12.71
precise-template lxc Vwi-a-tz- 7.81g lxc 12.67

Not only cloning the container was super fast, but it also barely used any new data in the ‘lxc’ thin pool.

The result of this work should soon make it’s way into Juju and, I hope, Docker, either via Alexander’s work or some other way. If you’re into Docker or a more hardcore LXC user, I hope you enjoy this feature and make good use of it.


2 thoughts on “Improving the state of art in container cloning

  1. Interesting…. I am going to try that out.
    Did you ever look into ploop of openvz :
    They put the container in a file (also thin) and have some intelligence in the ploop to perform something like snapshots and live migration.
    Looks like this could also be used with lxc.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.