Tip 1: Make shutdown work

The shutdown command does not work by default from virt-manager nor virsh. To fix that, install acpid on the template guest image:

sudo apt-get install acpid

After that, shutdown should work from both virt-manager and virsh. For some strange reason, reboot works only from virt-manager, not virsh.

Tip 2: Make serial console work

This is very handy for console over ssh (no need for virt-manager, nor any kind of VNC connection for console). See earlier post:

http://koo.fi/tech/2010/11/27/serial-console-for-ubuntu-server-1004-kvm-guests

Tip 3: Copy your ssh public key

It makes sense to copy your public ssh key at this point, so that you don't need to do it again for each guest. Simply go to your home directory on the template guest and run:

mkdir -p .ssh
chmod 700 .ssh
cd .ssh
cat >authorized_keys

Paste your public key there, and press Ctrl-D. Pasting does not work in a VNC console, but does work using the serial console trick mentioned in tip 2, or ssh connection.

Next, let's take a look at the problems mentioned in the start of the post:

Problem 1: Network interface naming

When Ubuntu boots up and detects a network interface with a MAC address which it has not seen before, it will take the next available eth number and add that to the udev rules file. The MAC addresses of network interfaces for cloned guests will be different from the template guest, so if you don't delete the corresponding lines from the file before cloning, the cloned guest's interface numbering will not start from eth0, but eth1 or something else depending on the number of interfaces you have configured.

In order to get your cloned guests' network interfaces start being numbered from eth0 onwards, simply delete the udev file which contains the rules for interface naming:

rm -f /etc/udev/rules.d/70-persistent-net.rules

That file will be regenerated at the first boot, so all clones will start their interface naming from eth0. But you must also remember that the next time you start up your template to make changes to it, it too will re-create the file. So remember to delete it every time you start up your template guest to make changes to it.

Problem 2: Individual SSH host keys

The ssh host keys are a bit more difficult thing. If you simply delete the keys from /etc/ssh/, they will not be regenerated automatically on Ubuntu. To make that happen, you need to run manually:

sudo dpkg-reconfigure openssh-server

In order to make that happen automatically during the first boot of a cloned guest, one line has to be added to the /etc/init/ssh.conf file (marked with bold):

/etc/init/ssh.conf:

respawn
respawn limit 10 5
umask 022
# replaces SSHD_OOM_ADJUST in /etc/default/ssh
oom never

pre-start script
    test -x /usr/sbin/sshd || { stop; exit 0; }
    test -e /etc/ssh/sshd_not_to_be_run && { stop; exit 0; }
    test -c /dev/null || { stop; exit 0; }

    # Regenerate ssh host keys if they are missing:
    test -f /etc/ssh/ssh_host_dsa_key || dpkg-reconfigure openssh-server

    mkdir -p -m0755 /var/run/sshd
end script

# if you used to set SSHD_OPTS in /etc/default/ssh, you can change the
# 'exec' line here instead
exec /usr/sbin/sshd

That will test the existence of host keys, and regenerate them if they are not present.

Before stopping your template guest, delete the ssh host keys:

rm -f /etc/ssh/*host*key*

Script to prepare template for cloning

I added a simple script, /usr/local/sbin/cloneprep, to ease the preparation of the template. It will simply delete your host ssh keys and udev persistent network rules (asking your permission first), and shut down the template guest afterwards so you can start cloning.

/usr/local/sbin/cloneprep:

#!/bin/sh

echo "TEMPLATE PREPARATION BEFORE CLONING"
echo
echo "This script will prepare this host for cloning. The network interface"
echo "name rules and ssh host keys will be deleted, but they will be"
echo "regenerated at the next boot (run sudo dpkg-reconfigure openssh-server"
echo "to regenerate ssh keys if needed)."
echo
echo "Are you sure you want to remove the ssh keys and network interface"
echo -n "rules on this host [y/N]? "

read YN

if [ "$YN" != "y" ]; then
  echo abort.
  exit 1
fi

echo
echo Removing persistent network interface rules...
rm -f /etc/udev/rules.d/70-persistent-net.rules

echo Removing host ssh keys...
rm -f /etc/ssh/*host*key*

echo
echo Your image is now ready for cloning. The machine will shut down.
echo

for i in 12 11 10 9 8 7 6 5 4 3 2 1; do echo "Halting machine in $i seconds (Ctrl-C to abort)"; sleep 1; done

halt

Give execute permission with "sudo chmod u+x /usr/local/sbin/cloneprep".

Remember to run cloneprep EVERY TIME you start up a template guest machine to change something. Don't use halt or shutdown, but run cloneprep instead.

It is not yet time to run it though, as we will do some further preparations to the template in the following section.

Problems 3 and 4: Hostname and IP address

To address these problems efficiently, we must create three of configuration files and a configuration script that will be run on each guest after the first boot.

To create a dozen guests, I created the following files under the template guest's /usr/local/etc.

guest01 192.168.1.101
guest02 192.168.1.102
guest03 192.168.1.103
guest04 192.168.1.104
guest05 192.168.1.105
guest06 192.168.1.106
guest07 192.168.1.107
guest08 192.168.1.108
guest09 192.168.1.109
guest10 192.168.1.110
guest11 192.168.1.111
guest12 192.168.1.112

(Use spaces instead of tabs in the cloneaddresses file. You can add as many spaces in between the names and addresses as you like, though.)

127.0.0.1   localhost
127.0.1.1   GUESTNAME

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
  address GUESTIP
  netmask 255.255.255.0
  gateway 192.168.1.1

I created the configuration script as /usr/local/sbin/cloneconf, and it looks like this:

/usr/local/sbin/cloneconf:

#!/bin/sh

echo "GUEST CONFIGURATION AFTER CLONING"
echo

GUESTNAME=""

if [ "$1" = "" ]; then
  echo -n "What is the hostname of this guest? "
  read GUESTNAME
else
  GUESTNAME="$1"
fi

GUESTIP="$(cat /usr/local/etc/cloneaddresses | egrep ^$GUESTNAME  | awk '{print $2}')"

if [ "$GUESTIP" = "" ]; then
  echo "Hostname not found. Available hostnames:"
  echo
  cat /usr/local/etc/cloneaddresses
  exit 1
fi

echo "This script will configure the host with the following settings:"
echo Hostname: $GUESTNAME
echo IP address: $GUESTIP
echo
echo /etc/hostname:
echo $GUESTNAME
echo
echo /etc/hosts:
cat /usr/local/etc/clonehosts | sed -e "s/GUESTNAME/$GUESTNAME/g"
echo
echo /etc/network/interfaces:
cat /usr/local/etc/cloneinterfaces | sed -e "s/GUESTIP/$GUESTIP/g"
echo
echo -n "Are you sure you want to continue [y/N]? "

read YN
if [ "$YN" != "y" ]; then
  echo abort.
  exit 1
fi

echo $GUESTNAME > /etc/hostname
cat /usr/local/etc/clonehosts | sed -e "s/GUESTNAME/$GUESTNAME/g" > /etc/hosts
cat /usr/local/etc/cloneinterfaces | sed -e "s/GUESTIP/$GUESTIP/g" > /etc/network/interfaces
test -f /etc/ssh/ssh_host_dsa_key || dpkg-reconfigure openssh-server

echo
echo Done. The machine will now be rebooted to make changes effective.
echo
for i in 12 11 10 9 8 7 6 5 4 3 2 1; do echo "Rebooting in $i seconds (Ctrl-C to abort)"; sleep 1; done
reboot

Give execute permission with "sudo chmod u+x /usr/local/sbin/cloneconf".

That script must be run individually on each cloned guest machine. Either give the hostname as the single argument to the script, or the script will ask you for the hostname if you don't do that. The hostname must be found in the cloneaddresses file. The rest of the settings will be determined from the configuration files.

Clone

One clone:

sudo virt-clone --original=template1 --name=guest01 --file=/var/lib/libvirt/images/guest01.img

Or a dozen clones:

for i in {01..12}; do sudo virt-clone -o template1 -n guest$i -f /var/lib/libvirt/images/guest$i.img; done

The virt-clone command will create a new xml configuration file under /etc/libvirt/qemu with a new UUID and randomly selected MAC address. Also, it will copy the image file to the location specified in the command line.

If you configured your template for serial console operation (link to the instructions in Tip 1 above), you can start up the new guest and get the console instantly using this command:

sudo virsh create /etc/libvirt/qemu/guest01.xml --console

To start your dozen guests simultaneously:

for i in {01..12}; do sudo virsh create /etc/libvirt/qemu/guest$i.xml; done

Then connect to the console of each individual guest with:

sudo virsh console guest01

You can get a list of running guests with "virsh list".

To end a serial console session, press Ctrl-]

Configure

The cloneconf script created in the first part of this post makes things easy.

After you have logged into a running guest machine (either VNC, serial console or ssh), run:

sudo cloneconf <hostname>

The hostname must be listed in the /usr/local/etc/cloneaddresses file created earlier. Otherwise the script will abort. It will show you changes it is about to make, and let you choose whether to commit them or not.

Reboot the guest after the script has run. Repeat for all clones.

The script will make changes to these files:

• /etc/hostname
• /etc/hosts
• /etc/network/interfaces

It will replace the hostname and IP address with a suitable value.