Vagrant tutorial

Bert Van Vreckem

LOADays, 5-6 April 2014

Whoami

Bert Van Vreckem

• Lecturer ICT at University College Ghent

– Mainly Linux & open source

– Coordinator Bachelor thesises

• @bertvanvreckem
• +BertVanVreckem
• [Link]
• [Link]
• [Link]

Have a question/remark? Please interrupt me!

Agenda

• Vagrant introduction
• Getting base boxes
• Con guring boxes
• Provisioning

– shell, Ansible, Puppet

– setting up a LAMP stack

• Creating base boxes

Introduction

What is Vagrant?

[Link]

• Written by Mitchell Hashimoto

• Command line tool
• Automates VM creation with

– VirtualBox
– VMWare

1
 – Hyper-V

• Integrates well with con guration management tools

– Shell
– Ansible
– Chef
– Puppet

• Runs on Linux, Windows, MacOS

Why use Vagrant?

• Create new VMs quickly and easily

– Only one command! vagrant up
• Keep the number of VMs under control
• Reproducability
• Identical environment in development and production
• Portability
– No more 4GB .ova les
– git clone and vagrant up

Assumptions

• Git
• Vagrant 1.5.1
• VirtualBox 4.3.10

– default Host-only network ([Link]/24)

• librarian-puppet

$ vagrant --version
Vagrant 1.5.1
$ VBoxHeadless --version
Oracle VM VirtualBox Headless Interface 4.3.10
(C) 2008-2014 Oracle Corporation
All rights reserved.

4.3.10r‘3012
$ ifconfig vboxnet0
=> 1‘[Link]

Try it yourself

• Clone the repository git clone git@[Link]’bertvv/[Link]

• When the slides mention checkpoint-nn , you can do git checkout tags/checkpoint-nn

2
Getting up and running

Minimal default setup:

$ vagrant init hashicorp/precise32

$ vagrant up
$ vagrant ssh

What happens under the hood?

$ vagrant init hashicorp/precise32

A Vagrant le is created (that’s all!)

What happens under the hood?

$ vagrant up
Bringing machine default up with virtualbox provider...
==> default’ Box hashicorp/precise32 could not be found. Attempting to find and install...
default’ Box Provider’ virtualbox
default’ Box Version’ >= 0
==> default’ Loading metadata for box hashicorp/precise32
default’ URL’ https’//[Link]/hashicorp/precise32
==> default’ Adding box hashicorp/precise32 (v1.0.0) for provider’ virtualbox
default’ Downloading’ https’//[Link]/hashicorp/precise32/version/1/provider/[Link]
==> default’ Successfully added box hashicorp/precise32 (v1.0.0) for virtualbox !
==> default’ Importing base box hashicorp/precise32 ...
==> default’ Matching MAC address for NAT networking...
==> default’ Checking if box hashicorp/precise32 is up to date...
==> default’ Setting the name of the VM’ example_default_13‘5‘‘6714768_3176
==> default’ Clearing any previously set network interfaces...
==> default’ Preparing network interfaces based on configuration...
default’ Adapter 1’ nat
==> default’ Forwarding ports...
default’ 22 => 2222 (adapter 1)

==> default’ Booting VM...

==> default’ Waiting for machine to boot. This may take a few minutes...
default’ SSH address’ [Link]’2222
default’ SSH username’ vagrant
default’ SSH auth method’ private key
==> default’ Machine booted and ready!
==> default’ Checking for guest additions in VM...
default’ The guest additions on this VM do not match the installed version of
default’ VirtualBox! In most cases this is fine, but in rare cases it can
default’ prevent things such as shared folders from working properly. If you see
default’ shared folder errors, please make sure the guest additions within the
default’ virtual machine match the version of VirtualBox you have installed on

3
 default’ your host and reload your VM.
default’
default’ Guest Additions Version’ 4.2.0
default’ VirtualBox Version’ 4.3
==> default’ Mounting shared folders...
default’ /vagrant => /home/bert/CfgMgmt/vagrant-example

What happens under the hood?

$ vagrant init hashicorp/precise32

• The base box is downloaded and stored locally

– in ~/.vagrant.d/boxes/

• A new VM is created and con gured with the base box as template
• The VM is booted
• The box is provisioned

– only the rst time, must be done manually afterwards

Done!

You now have a working VM, ready for use:

$ vagrant ssh
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic-pae i686)

* Documentation’ https’//[Link]/
Welcome to your Vagrant-built virtual machine.
Last login’ Fri Sep 14 06’22’31 2012 from [Link]
vagrant@precise32’~$

Con guring Vagrant boxes

Vagrant le

Minimal Vagrant le (checkpoint-01):

VAGRANTFILE_API_VERSION = 2

[Link](VAGRANTFILE_API_VERSION) do |config|
[Link] = hashicorp/precise32
end

Vagrant le = Ruby

. . .

This is Ubuntu 12.04 LTS 32 bit,

Let’s say we want CentOS 6.5 64 bit

4
 Finding base boxes

• [Link] (since 1.5)

• [Link] (pre-1.5 boxes)

Using another base box

From the command line (Vagrant cloud):

$ vagrant init alphainternational/centos-6.5-x64

. . .

From the command line ( old , pre-1.5 style):

$ vagrant box add --name centos65 \

http’//[Link]/vagrant-boxes/[Link]
$ vagrant init centos65

. . .

In your Vagrant le (only applies to old style):

VAGRANTFILE_API_VERSION = 2

[Link](VAGRANTFILE_API_VERSION) do |config|
[Link] = centos65
[Link].box_url =
http’//[Link]/vagrant-boxes/[Link]
end

Applying the change

$ vagrant destroy
default’ Are you sure you want to destroy the default VM? [y/N] y
==> default’ Forcing shutdown of VM...
==> default’ Destroying VM and associated drives...
$ vagrant up
[...]
$ vagrant ssh

Con guring the VM

(checkpoint-02)

1 VAGRANTFILE_API_VERSION = 2
2

3 HOST_NAME = box001
4

5 [Link](VAGRANTFILE_API_VERSION) do |config|

5
 6

7 [Link] = HOST_NAME
8 [Link] = alphainternational/centos-6.5-x64
9 [Link] ’private_network,
10 ip’ 1‘[Link] ,
11 netmask’ [Link]
12

13 [Link] ’virtualbox do |vb|

14 [Link] = HOST_NAME
15 [Link] [ modifyvm , ’id, --memory , 256]
16 end
17 end

Con guring the VM

For more info,

• see the docs at [Link]

• or the default Vagrantfile

Applying changes

When you change the Vagrantfile, do:

$ vagrant reload

Or, if the change is profound:

$ vagrant destroy -f
$ vagrant up

Setup with multiple VMs

Vagrant le:

[Link] HOST_NAME do |node|

[Link] = HOST_NAME
[...]
end

Specify HOST_NAME after vagrant command:

$ vagrant status # Status of *all* boxes

$ vagrant up box001 # Boot box001
$ vagrant up # Boot *all* defined boxes
$ vagrant ssh box001

6
 Setup with multiple VMs: Example

(checkpoint-03)

1 VAGRANTFILE_API_VERSION = 2
2

3 [Link](VAGRANTFILE_API_VERSION) do |config|
4

5 [Link] box001 do |node|

6 [Link] = box001
7 [Link] = alphainternational/centos-6.5-x64
8 [Link] ’private_network,
9 ip’ 1‘[Link] ,
10 netmask’ [Link]
11

12 [Link] ’virtualbox do |vb|

13 [Link] = box001
14 end
15 end

Setup with multiple VMs: Example (cont’d)

16 [Link] box002 do |node|

17 [Link] = box002
18 [Link] = alphainternational/centos-6.5-x64
19 [Link] ’private_network,
20 ip’ 1‘[Link] ,
21 netmask’ [Link]
22

23 [Link] ’virtualbox do |vb|

24 [Link] = box002
25 end
26 end
27 end

Setup with multiple VMs: Example (cont’d)

Don’t repeat yourself! (checkpoint-04)

1 hosts = [ { name’ box001 , ip’ 1‘[Link] },

2 { name’ box002 , ip’ 1‘[Link] }]
3

4 [Link](VAGRANTFILE_API_VERSION) do |config|
5 [Link] do |host|
6 [Link] host[’name] do |node|
7 [Link] = host[’name]
8 [Link] = alphainternational/centos-6.5-x64
9 [Link] ’private_network,
10 ip’ host[’ip],
11 netmask’ [Link]
12 [Link] ’virtualbox do |vb|

7
13 [Link] = host[’name]
14 end
15 end
16 end
17 end

Summary

$ vagrant init user/box # Create Vagrantfile for specified base box

$ vim Vagrantfile # Customize your box
$ vagrant up [host] # Create VM(s) if needed and boot
$ vagrant reload [host] # After every change to Vagrantfile
$ vagrant halt [host] # Poweroff
$ vagrant destroy [host] # Clean up!
$ vagrant ssh [host] # log in
$ vagrant status [host] # Status of your VM(s)

Provisioning

Provisioning

= From Just Enough Operating System to fully functional con gured box

• Shell script
• Ansible
• Puppet (Apply + Agent)
• Chef (Solo + Client)
• Docker
• Salt

Shell provisioning

Shell provisioning

Add to your Vagrant le

[Link] shell , path’ [Link]

Put the script into the same folder as Vagrantfile

Recommended work ow

• First do the installation manually (vagrant ssh)

• Make sure every command runs without user interaction!
• Record every command in the script
• If everything works: vagrant destroy -f && vagrant up

8
Provisioning script

(checkpoint-05)

Installs Apache and PHP

#!/bin/bash -eu
# [Link] -- Install Apache and a test PHP script

sudo rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6

yum install -y httpd php

service httpd start

chkconfig httpd on

cat > /var/www/html/[Link] << EOF

<?php phpinfo(); ?>
EOF

MySQL is left as an exercise for the reader ;-)

Synced folders

(checkpoint-06)

• Add to your Vagrantfile:

[Link].synced_folder html , /var/www/html

• Create folder html in your project root

$ tree
.
|-- html
| ‘-- [Link]
|-- [Link]
‘-- Vagrantfile

• Vagrant reload

Disadvantages of shell provisioning

• Not very exible

• Script should be non-interactive
• Not scalable
– Long Bash scripts are horrible!
• Idempotence not guaranteed
– What happens when you run provision script multiple times?
– Change to script is expensive: vagrant destroy && vagrant up

9
 Provisioning with Ansible

Ansible

[Link]

• Con guration management tool written in Python

• Simple con guration (YAML)
• No agent necessary (but recommended for large setups)
• Idempotent

. . .

(of course, you know this, you went to the talks yesterday…)

Vagrant con guration

[Link] box001 do |node|

[...]
[Link] ansible do |ansible|
[Link] = ansible/[Link]
end
end

Pro tips:

• define directive is important to make automatic inventory work

– See Vagrant/Ansible documentation

• try to mimic standard Ansible directory structure

– See Ansible best practices

Let’s build a LAMP stack!

First, on one box

Then, database on a separate machine

Vagrant le

(checkpoint-07)

1 VAGRANTFILE_API_VERSION = 2
2 hosts = [ { name’ box001 , ip’ 1‘[Link] },
3 { name’ box002 , ip’ 1‘[Link] } ]
4

5 [Link](VAGRANTFILE_API_VERSION) do |config|
6 [Link] = alphainternational/centos-6.5-x64
7 [Link] do |host|

10
 8 [Link] host[’name] do |node|
9 [Link] = host[’name]
10 [Link] ’private_network,
11 ip’ host[’ip],
12 netmask’ [Link]
13 [Link].synced_folder html , /var/www/html
14

15 [Link] ’virtualbox do |vb|

16 [Link] = host[’name]
17 end
18

19 [Link] ansible do |ansible|

20 [Link] = ansible/[Link]
21 end
22 end
23 end
24 end

Ansible project structure

$ tree ansible/
ansible/
|-- group_vars
| -- all
|-- roles
| |-- common
| | -- tasks
| | -- [Link]
| |-- db
| | -- tasks
| | -- [Link]
| -- web
| -- tasks
| -- [Link]
-- [Link]

Main Ansible con g le: [Link]

---
- hosts’ box001
sudo’ true
roles’
- common
- web
- db

Common role

---
# file common/tasks/[Link]

11
 - name’ Install base packages
yum’ pkg={{item}} state=installed
with_items’
- libselinux-python

Web role

---
# file web/tasks/[Link]
- name’ Install Apache
yum’ pkg={{item}} state=installed
with_items’
- httpd
- php
- php-xml
- php-mysql

- name’ Start Apache service

service’ name=httpd state=running enabled=yes

Db role

1 ---
2 # file db/tasks/[Link]
3 - name’ Install MySQL
4 yum’ pkg={{item}} state=installed
5 with_items’
6 - mysql
7 - mysql-server
8 - MySQL-python
9

10 - name’ Start MySQL service

11 service’ name=mysqld state=running enabled=yes
12

13 - name’ Create application database

14 mysql_db’ name={{ dbname }} state=present
15

16 - name’ Create application database user

17 mysql_user’ name={{ dbuser }} password={{ dbpasswd }}
18 priv=*.*’ALL host= localhost state=present

Variables

---
# file group_vars/all

# Application database
dbname’ appdb
dbuser’ appusr
dbpasswd’ CaxWeikun6

12
Work ow

1. Write Vagrantfile

• vagrant up and vagrant reload until you get it right

2. Write con guration

• vagrant provision until you get it right

3. Think you’re done?

• vagrant destroy -f and vagrant up

Install a webapp

E.g. Mediawiki

1. Unpack latest [Link] into html/wiki/ directory

2. Surf to [Link] and follow instructions
3. Enter values from group_vars/all in the install page
4. Download [Link] and save in html/wiki/

Automating Mediawiki installation is left as an exercise to the reader… ;-)

How to use this for production

Inventory le, automatically created by Vagrant:

$ cat .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory
# Generated by Vagrant

box001 ansible_ssh_host=[Link] ansible_ssh_port=2222

box002 ansible_ssh_host=[Link] ansible_ssh_port=2200

In production, just use a di erent inventory le!

Move database to another box

(checkpoint-08)

What should change?

. . .

---
# file [Link]
- hosts’ box001
sudo’ true
roles’
- common
- web

13
- hosts’ box002
sudo’ true
roles’
- common
- db

Move database to another box (cont’d)

What should change?

---
# db/tasks/[Link]
[...]
- name’ Create application database user
mysql_user’ name={{ dbuser }} password={{ dbpasswd }}
priv=*.*’ALL host= % state=present

This should be easy to automate

Provisioning with Puppet

Puppet

[Link]

• One of the market leaders in con guration management

• Has its own con guration language
• Many reusable modules available
• Needs an agent on hosts under control
• Usually set up with a central server (puppet master)
• Puppet should be already on your base box!

. . .

Do I have to introduce Puppet at all?

Vagrant con guration

[Link] HOST_NAME do |node|

[Link].synced_folder puppet , /etc/puppet
[Link] puppet do |puppet|
puppet.manifests_path = puppet/manifests
puppet.manifest_file = [Link]
end
end

Pro tips:

14
 • The synced_folder directive makes Puppet just work

– No other directives needed (e.g. module_path, manifest_path)

– Installing les outside of modules
– Same [Link] for Vagrant and production
– Easier to reuse in production environment

• Mimic Puppet directory structure and best practices

Let’s build a LAMP stack!

Vagrant le

(checkpoint-0‘)

1 VAGRANTFILE_API_VERSION = 2
2 HOST_NAME = box001
3 DOMAIN = [Link]
4 HOST_IP = 1‘[Link]
5

6 [Link](VAGRANTFILE_API_VERSION) do |config|
7 [Link] = alphainternational/centos-6.5-x64
8 [Link] HOST_NAME do |node|
9 [Link] = ”#{HOST_NAME}.#{DOMAIN}”
10 [Link] ’private_network,
11 ip’ HOST_IP,
12 netmask’ [Link]
13 [Link].synced_folder html , /var/www/html
14 [Link].synced_folder puppet , /etc/puppet

Vagrant le (cont’d)

1 [Link] ’virtualbox do |vb|

2 [Link] = HOST_NAME
3 [Link] [ modifyvm , ’id, --memory , 256]
4 end
5

6 [Link] puppet do |puppet|

7 puppet.manifests_path = puppet/manifests
8 puppet.manifest_file = [Link]
9 end
10 end
11 end

Puppet project structure

$ tree -I modules --prune puppet/

puppet/
|-- manifests
| |-- nodes
| | |-- [Link]

15
| | -- [Link]
| -- [Link]
-- Puppetfile

Main Puppet les

# file manifests/[Link]

# Load node definitions

import nodes/*

# file manifests/nodes/[Link]

node default {
notice(”I m node ${’’hostname} with IP ${’’ipaddress_eth1}”)

Managing 3rd party modules

Here, we use librarian-puppet

# Puppetfile -- Configuration for librarian-puppet

# Bootstrap by running librarian-puppet init

forge ”http’//[Link]”

mod ”puppetlabs/stdlib”
mod ”puppetlabs/concat”

mod ”puppetlabs/apache”
mod ”puppetlabs/mysql”

Working with Git submodules is also common, e.g.

$ git submodule add git@[Link]’puppetlabs/[Link] modules/mysql

$ cd modules/mysql
$ git checkout tags/2.2.3

De nition of box001

# file manifests/nodes/[Link]

node box001 inherits default {

# Apache and PHP
class { apache ’ }
class { apache’’mod’’php ’ }

package { [ php-mysql , php-xml ]’

16
 ensure => installed,
}

# MySQL
include ’’mysql’’server

mysql’’db { appdb ’
user => dbusr ,
password => vaygDeesh1 ,
host => localhost ,
}

Development vs Production

(checkpoint-10)

How to handle di erences between development and production?

Puppet’s answer: Hiera

Hiera con guration

puppet/[Link]:

---
’backends’
- yaml
’hierarchy’
- %{’’environment}/%{’’clientcert}
- common
’yaml’
’datadir’ /etc/puppet/hiera

$ tree puppet/hiera
puppet/hiera
|-- [Link]
|-- development
| -- [Link]
-- production
-- [Link]

Hiera data

---
# file hiera/[Link]
mysql’’host’ localhost

---
# puppet/hiera/development/[Link]

17
 mysql’’appdb’ appdb
mysql’’user’ dbusr
mysql’’password’ letmein

---
# file puppet/hiera/production/[Link]
mysql’’appdb’ db72437
mysql’’user’ u440380
mysql’’password’ ifwoHaffEtHafwivIj7

Using Hiera data

Vagrantfile’

[Link] puppet do |puppet|

puppet.manifests_path = puppet/manifests
puppet.manifest_file = [Link]
[Link] = [ --environment development ]
end

puppet/manifests/nodes/[Link]

$appdb = hiera( mysql’’appdb )

mysql’’db { $appdb’
user => hiera( mysql’’user ),
password => hiera( mysql’’password ),
host => hiera( mysql’’host ),
}

Best practices

Best practices

• Follow guidelines of CfgMgmt tool

– so you can use your box outside of Vagrant
• Keep Vagrantfile minimal
– change Vagrantfile => vagrant reload
– more expensive than vagrant provision

Vagrantfile bloat

1 # Enable provisioning with chef solo

2 [Link] ’chef_solo do |chef|
3 chef.cookbooks_path = ”cookbooks”
4 chef.add_recipe ”yum”
5 chef.add_recipe ”yum’’epel”
6 chef.add_recipe ”openssl”

18
 7 chef.add_recipe ”apache2”
8 chef.add_recipe ”apache2’’default”
9 chef.add_recipe ”apache2’’mod_ssl”
10 chef.add_recipe ”mysql”
11 chef.add_recipe ”mysql’’server”
12 chef.add_recipe ”php”
13 chef.add_recipe ”php’’module_apc”
14 chef.add_recipe ”php’’module_curl”
15 chef.add_recipe ”php’’module_mysql”
16 chef.add_recipe ”apache2’’mod_php5”
17 chef.add_recipe ”apache2’’mod_rewrite”
18 [Link] = {
19 ’mysql => {
20 ’server_root_password => root ,
21 ’bind_address => [Link]
22 }
23 }
24 end

Creating base boxes

Creating base boxes

Sometimes, the available base boxes just aren’t good enough…

Manually

1. Create a VM, and take some requirements into account

• a.o. vagrant user with sudo, ssh, package manager, Guest Additions
• if you want: Puppet, Chef, …

2. Execute vagrant package –base my-vm

• Result: le [Link]

Disadvantages

• It’s manual
• Not quite reproducable for other provider (e.g. VMWare, Hyper-V, bare metal)

Enter Packer

[Link]

Packer is a tool for creating identical machine images for multiple platforms from a single source con guration.

19
Packer template

• JSON le with settings

– e.g. ISO download URL, VM type, provisioner

• Kickstart le

– Automates installation from ISO

• Post-installation scripts

– e.g. Con gure for Vagrant, install Puppet, clean up yum repository, zerodisk (smaller disk images)

• Find loads of Packer templates at [Link]

– Cr*p, only for Chef & Salt…

That’s it!

What I didn’t cover

• Provisioning with Chef

• Security (SELinux, rewall)
• Testing

Thank you!

Presentation slides: [Link]

Code: [Link]

More at:

[Link] [Link] [Link]

@bertvanvreckem

Figure 1: CC-BY

20