Benjamin Sago / ogham / cairnrefinery / etc…

Technical notes › Create Vagrant boxes with Packer

Spinning up a Vagrant machine usually involves starting from a base Debian or Ubuntu image, then provisioning it with shell scripts or something like Ansible or Chef. Depending on how much there is to provision, this can end up taking quite a bit of time, which means there’s more of a downside to halting or destroying a running machine to save on disk space. If Vagrant development enviroments are meant to be truly disposable, and re-buildable, they should be easy to dispose and re-build.

The recommended solution for this is to “pre-bake” your environment into a box, and then reference this box in your Vagrantfile. This means that if you ever need to tear down and re-create your image, or if you want to have multiple machines that all use the same base image, you can save yourself some provisioning time.

One way to do this is with Packer, an image builder also from HashiCorp; specifically, the Vagrant builder for Packer. It should come as no surprise that two HashiCorp tools are able to work together in this way.

Creating a Vagrant box

Using Packer, you can create a box out of a VMware image, a Virtualbox image, an ISO, or plenty of other things. However, the process for doing so is cumbersome, especially if you have to automate the OS installation process.

For this reason, I recommend installing one of the boxes from Chef’s Bento project. They regularly build updated versions of popular Linux distributions and Unix systems, and upload the resulting boxes to Vagrant Cloud. You can use these boxes to build upon yourself.

Here is an example Packer file, which creates a Vagrant box by starting with Ubuntu 20.10, launching a VM, executing a shell script on it, and saving the resulting filesystem as another box in the target/ directory:

vagrant-dev-image.pkr.hcl
source "vagrant" "ubuntu" {
  provider = "vmware_desktop"
  communicator = "ssh"

  source_path = "bento/ubuntu-20.10"
  box_version = "202107.08.0"

  output_dir = "target"
  add_force = true
}

build {
  sources = ["source.vagrant.ubuntu"]

  provisioner "shell" {
    script = "install-dependencies.sh"
  }
}

The shell script can be anything you like:

install-dependencies.sh
trap 'exit' ERR

# Install general build dependencies
echo "Installing tools"
sudo apt-get update -qq
sudo apt-get install -qq -o=Dpkg::Use-Pty=0 \
    build-essential libssl-dev pkg-config git zip

# Make image smaller
echo "Freeing a bit of disk space"
sudo apt-get clean
sudo apt-get -qq autoremove

You can then run this build using the packer build command:

$ packer build vagrant-dev-image.pkr.hcl
vagrant.ubuntu: output will be in this color.

==> vagrant.ubuntu: Creating a Vagrantfile in the build directory...
==> vagrant.ubuntu: Adding box using vagrant box add ...
    vagrant.ubuntu: (this can take some time if we need to download the box)
==> vagrant.ubuntu: Calling Vagrant Up (this can take some time)...
==> vagrant.ubuntu: Using SSH communicator to connect: 127.0.0.1
==> vagrant.ubuntu: Waiting for SSH to become available...
==> vagrant.ubuntu: Connected to SSH!
==> vagrant.ubuntu: Provisioning with shell script: install-dependencies.sh
    vagrant.ubuntu: Installing tools
==> vagrant.ubuntu: Packaging box...
==> vagrant.ubuntu: destroying Vagrant box...
Build 'vagrant.ubuntu' finished after 1 minute 36 seconds.

==> Wait completed after 1 minute 36 seconds

==> Builds finished. The artifacts of successful builds are:
--> vagrant.ubuntu: Vagrant box 'package.box' for 'vmware_desktop' provider

Packer will write the box into the target/ directory, as per our instructions. We can then vagrant add the box file, and the box with that name will be available for us to use in our Vagrantfiles.

Troubleshooting

Unfortunately, the error diagnostics for using Packer in this way are… not great. If there is an error on the Vagrant side of things that Packer does not know how to handle, it will spit out this error:

Build '<vagrant.ubuntu>' errored after 2 seconds 463 milliseconds:
error: SSH Port was not properly retrieved from SSHConfig.

This could mean that there is a syntax error in the Ruby syntax of the Vagrantfile, or that the base box you are using is not supported by your provider, or one of many other things. If you see it, your best bet is not to assume that there’s something wrong with your SSH config or whatever. Instead, you should capture the Vagrantfile that Packer is using (with packer build -debug) and trying to use it with Vagrant manually.

Giving more cores to the VM

The first box I used this for was going to be for Rust development: one that contained several versions of the compiler along with some associated build tools. But when I ran the installation script for the first time, I noticed it was much, much slower than running locally.

Upon closer inspection, Packer was using the VM provider’s default values for how many CPU cores and how much memory to give the machine: a measly two cores and a paltry two gigabytes! If you have a faster machine, you can afford to increase these numbers.

The instructions for how to do this are badly documented on the Packer website, but it’s possible. To do this, first add a template field to the source block of your Packer file:

vagrant-dev-image.pkr.hcl
source "vagrant" "ubuntu" {
  # ...
  source_path = "bento/ubuntu-20.10"
  template = "Vagrantfile.tpl"
  # ...
}

Then, create a Vagrantfile based off the default one, and include config.vm.provider blocks to set the amount of memory and number of processors:

Vagrantfile.tpl
Vagrant.configure(2) do |config|
  config.vm.define "source", autostart: false do |source|
    source.vm.box = "{{.SourceBox}}"
    config.ssh.insert_key = {{.InsertKey}}
  end

  config.vm.define "output" do |output|
    output.vm.box = "{{.BoxName}}"
    output.vm.box_url = "file://package.box"
    config.ssh.insert_key = {{.InsertKey}}
  end

  # These ‘provider’ definitions are not present in the original template.
  config.vm.provider "virtualbox" do |v|
    v.memory = 4096
    v.cpus = `nproc`.to_i
  end

  config.vm.provider "vmware_desktop" do |v|
    v.vmx['memsize'] = 4096
    v.vmx['numvcpus'] = `nproc`.to_i
  end

  {{if ne .SyncedFolder "" -}}
    config.vm.synced_folder "{{.SyncedFolder}}", "/vagrant"
  {{- else -}}
    config.vm.synced_folder ".", "/vagrant", disabled: true
  {{- end}}
end

As you might be able to spot from the weird templating syntax at the end of the file, this is not Ruby — it’s a Go template, executed by Packer, that outputs the Vagrantfile when run. The Ruby code uses `nproc`.to_i to grab the number of available processors at runtime, and starts the VM with the configured number of CPU cores and amount of memory available. If you’re doing a lot of CPU-heavy operations during provisioning (such as compiling development tools from source), this should significantly cut down on the amount of time your boxes take to build.∎

All tech notes