Since early August I’ve been looking at improving the state of the art in container cloning with an ultimate goal of making Juju faster when used in a local provider context, which happens to be where I spend most of my days lately. For those who don’t know, the local provider in Juju uses LXC with the LXC-provided ‘ubuntu-cloud’ template in order to provide an environment that’s as similar as possible to provisioning a cloud instance elsewhere.
In looking at the various storage backends supported by LXC and experimenting with each of them, I’ve stumbled on various issues, from broken inotify support in overlayfs to random timing issues deleting btrfs snapshots. Eventually I’ve discovered the holy grail of LVM Thin Provisioning and started working on a patch to LXC which would allow it to create volumes in a thin provisioned pool. In the meantime, Docker announced their intent of adding support for LVM Thin Provisioning too. I’m happy to announce that the work I started in August is now merged into LXC as various separate pull requests (#67, #70, #72, #74) and is already available in the LXC Daily PPA. I’ve even created a Gist showing how to use it.
As pointed out by a colleague today, Thin Provisioning support should soon land in Docker itself. I applaud the initiative, and am really looking forward to seeing this feature land in Docker. I wish though there was a little more coordination going on to get that work upstreamed into LXC instead, to the benefit of everyone. Regardless, I’m committed to make sure that the differences between whatever ends up landing in Docker and what I’ve added to LXC eventually converge. One such difference is that I’ve simply shelled out to the ‘lvs’ command while Alexander is using libdevmapper directly, something I’ve looked at doing but felt a little over my head. I’ve also haven’t got around to making the filesystem default to ext4 with DISCARD support yet, but that’s on the top of my list.
So without much ado, let’s look at an example with a loopback-mounted device backed by a sparse-allocated file.
$ sudo fallocate -l 6G /tmp/lxc $ sudo losetup -f /tmp/lxc $ sudo pvcreate /dev/loop0 $ sudo vgcreate lxc /dev/loop0
$ sudo lvcreate -l 95%FREE --type thin-pool --thinpool lxc lxc $ sudo lvs LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert lxc lxc twi-a-tz- 5.70g 0.00
$ sudo lxc-create -n precise-template -B lvm --fssize 8G -t ubuntu-cloud -- -r precise $ sudo lvs LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert lxc lxc twi-a-tz- 5.70g 17.38 precise-template lxc Vwi-a-tz- 7.81g lxc 12.67
$ time sudo lxc-clone -s precise-template precise-copy real 0m0.276s user 0m0.021s sys 0m0.085s
Plenty fast I say. What does the newly created volume looks like?
$ sudo lvs LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert lxc lxc twi-a-tz- 5.70g 17.44 precise-copy lxc Vwi-a-tz- 7.81g lxc precise-template 12.71 precise-template lxc Vwi-a-tz- 7.81g lxc 12.67
Not only cloning the container was super fast, but it also barely used any new data in the ‘lxc’ thin pool.
The result of this work should soon make it’s way into Juju and, I hope, Docker, either via Alexander’s work or some other way. If you’re into Docker or a more hardcore LXC user, I hope you enjoy this feature and make good use of it.
2 thoughts on “Improving the state of art in container cloning”
Interesting…. I am going to try that out.
Did you ever look into ploop of openvz : http://openvz.org/Ploop
They put the container in a file (also thin) and have some intelligence in the ploop to perform something like snapshots and live migration.
Looks like this could also be used with lxc.
First I heard about it. Seems pretty cool!