Bake vs. Fry

Pro tip: combine both

• Generate a custom image on a schedule

• Finish configuration at boot-time (updates, config files, etc.)

• No need to regenerate everything each time a VM is run
• The image can serve as a minimal starting point for several applications
  • Without useless componants

Customize an AMI

An AMI: appliance + meta-data

• a unique identifier in a region
• multiple kinds:
  • Public
  • Paid (marketplace)
  • Private

User data

• Can be data or a script
• Limited to 16 KB
• Can be base64-encoded
• Accessible via HTTP from the instance

curl http://169.254.169.254/latest/meta-data/user-data

Boxgrinder

Principle

• YAML description
• Generates multiple appliance formats (KVM, VMWare, Xen, EC2, VirtualBox)
• Deployment (EC2, OpenStack, libvirt)
• Test VMs locally before deploying them in your cloud

Support

• Supports mainly RedHat-based distributions
• But: cross-OS builds
  • “Build a CentOS or RHEL from a Fedora Host”
• Community plugins
  • Ubuntu: https://github.com/rubiojr/boxgrinder-ubuntu-plugin

Boxgrinder

Usage

• Create a KVM appliance boxgrinder-build centos.appl

• Create an appliance and deploy run it in VirtualBox boxgrinder-build centos.appl -p virtualbox -d local

Usage

• Create an appliance and run it via libvirt:

boxgrinder-build definition.appl \
    -d libvirt --delivery-config \
    connection_uri:qemu:///system,\
    image_delivery_uri:/var/lib/libvirt/images

AWS Configuration

2 sections in $HOME/.boxgrinder/config

S3

access_key: <ACCESS_KEY>
secret_access_key: <SECRET_ACCESS_KEY>
bucket: <BUCKET>
account_number: <ACCOUNT_ID> # see security credentials page
path: /
cert_file: /home/me/.boxgrinder/cert-2GDSYWYSIVXTZIDBLYZTAOX4KW.pem
key_file: /home/me/.boxgrinder/pk-2GFVVWYSIVXTZIDBLYZTAOX4KW.pem
region: eu-west-1

EBS

access_key: <ACCESS_KEY>
secret_access_key: <SECRET_ACCESS_KEY>
account_number: <ACCOUNT_ID>
delete_on_termination: true
region: eu-west-1
availability_zone: eu-west-1a

Deploying on EC2

• Deploy an instance-store AMI (S3)

boxgrinder-build centos58.appl -p ec2 -d ami

• Deploy an EBS AMI

boxgrinder-build centos63.appl \
    -p ec2 -d ebs \
    --delivery-config overwrite:true,\
    destroy_instances:true

Appliance file

name: "centos58-#BASE_ARCH#-jeos"
summary: CentOS 5 Core
version: 1
release: 0

os:
  name: centos
  version: 5 
  password: centos

hardware:
  partitions:
    "/":
      size: 8

Ubuntu VM-Builder

Principle

Create Ubuntu-based appliances

• Also supports Debian testing
• Supports:
  • Hypervisors: KVM, Xen, VirtualBox, VMWare
  • Deployment: libvirt, EC2

Installation

apt-get install ubuntu-vm-builder python-vm-builder-ec2

Generate an appliance

General syntax:
vmbuilder <hypervisor> <distribution>

Generate and deploy an appliance

vmbuilder kvm ubuntu --suite lucid \
    --arch amd64 \
    --libvirt qemu://system

Generate an appliance (2)

• With a pre-configured user
  vmbuilder kvm ubuntu --user bob --password toto

• With installed packets

vmbuilder kvm ubuntu --addpkg apache2 \
    --addpkg libapache2-mod-php5 \
    --addpkg php5-mysql 

Create an appliance and upload it to S3

sudo vmbuilder xen ubuntu --suite=karmic --ec2 \
  --ec2-cert=<ec2 cert> \
  --ec2-key=<ec2 key> \
  --ec2-access-key=<aws access key> \
  --ec2-secret-key=<aws secret key> \
  --ec2-user=<aws # number> \
  --ec2-bucket=<ec2 bucket name> \
  --ec2-prefix=<image prefix> \
  --ec2-version="Description of your EC2 image" \
  --firstboot=/usr/share/doc/python-vm-builder-ec2/examples/ec2-firstboot.sh \
  --part=/usr/share/doc/python-vm-builder-ec2/examples/ec2-<arch>-part-file.txt

Cloud-init

Cloud-init

• Developped by Canonical (doc)
• Fetches user-data
• Works arround the size limit
  • Preconfigured actions
  • Compression
  • Concatenation of resources (mime-type encoding)
• Pre-installed on official Amazon AMIs

Some actions

• Set the locale
• Set the hostname
• Generate a key-pair
• Add keys to $HOME/.ssh/authorized_keys
• Mount volumes

#Cloud-config example

#cloud-config
package_update : true 
package_upgrade : true
packages :
  - apache2
apt_sources : 
  - source :deb http://archive.ubuntu.com/ubuntu karmic main …
ssh_authorized_keys :
  - ssh-rsa "<ssh public key>"
locale : fr_FR.UTF-8
runcmd :
  - echo "hello world !" > /home/ubuntu/message

Include example

#include
http://emn.fr/download/ec2-config.txt
ftp://emn.fr/download/vhost.conf

Generating a multipart user-data file

Using the write-mime-multipart utility (package with cloud-init):

write-mime-multipart –output=combined-userdata.txt \
   my-include.txt:text/x-include-url \
   my-user-script.txt:text/x-shellscript \
   my-cloudconfig.txt

Windows AMIs

Prepare to suffer

• Creating a Windows AMI from scratch is a delicate process
• Recommanded method:
  • Run a Windows instance
  • Customize it by hand
  • Create a bundle from the CLI or Web Console
  • Save it
  • http://docs.aws.amazon.com/workspaces/latest/adminguide/bundle_tutorial.html
• Possibility to create the initial setup with the EC2Config service (PowerShell)
• Rather long process (expect no less than 15 minutes to get the admin password)

How-to: short version

• CLI or Web console
• Applies to other OS as well

EBS images

• Create an image from an EBS: euca-create-image -n <name> <instance-id>
• Make the image public: euca-modify-image-attribute <ami-id> -l -a all
• Or private: euca-modify-image-attribute <ami-id> -l -r all
• Delete the image euca-deregister <ami-id>

Instance-store images

• Save the image in S3: euca-bundle-instance <id> --bucket <bucket> --prefix <bundle-name>
• Register it as an AMI: euca-register <bucket>/image.manifest.xml -n <ami-name>