The longer I work in IT the more I dislike repetitive processes. For example, when updating my Oracle Linux 8 Vagrant Base Box I repeat the same process over and over:

• Boot the VirtualBox (source) VM
• Enable port forwarding for SSH
• SSH to the VM to initiate the update via dnf update -y && reboot
• Run vagrant package, calculate the SHA256 sum, modify the metadata file
• Use vagrant box update to make it known to vagrant

There has to be a better way to do that, and in fact there is. A little bit of shell scripting later all I need to do is run my “update base box” script, and grab a coffee while it’s all done behind the scenes. The most part of the exercise laid out above is quite boring, but I thought I’d share how I’m modifying the metadata file in the hope to save you a little bit of time and effort. If you would like a more thorough explanation of the process please head over to my previous post.

Updating the Metadata File

If you would like to version-control your vagrant boxes locally, you need a metadata file, maybe something similar to ol8.json shown below. It defines my Oracle Linux 8 boxes (at the moment there is only one):

$ cat ol8.json 
{
  "name": "ol8",
  "description": "Martins Oracle Linux 8",
  "versions": [
    {
      "version": "8.4.0",
      "providers": [
        {
          "name": "virtualbox",
          "url": "file:///vagrant/boxes/ol8_8.4.0.box",
          "checksum": "b28a3413d33d4917bc3b8321464c54f22a12dadd612161b36ab20754488f4867",
          "checksum_type": "sha256"
        }
      ]
    }
  ]
}

For the sake of argument, let’s assume I want to upgrade my Oracle Linux 8.4.0 box to the latest and greatest packages that were available at the time of writing. As it’s a minor update I’ll call the new version 8.4.1. To keep the post short and (hopefully) entertaining I’m skipping the upgrade of the VM.

Option (1): jq

Fast forward to the metadata update: I need to add a new element to the versions array. I could have used jq for that purpose and it would have been quite easy:

$ jq '.versions += [{
>       "version": "8.4.1",
>       "providers": [
>         {
>           "name": "virtualbox",
>           "url": "file:///vagrant/boxes/ol8_8.4.1.box",
>           "checksum": "ecb3134d7337a9ae32c303e2dee4fa6e5b9fbbea5a38084097a6b5bde2a56671",
>           "checksum_type": "sha256"
>         }
>       ]
>     }]' ol8.json
{
  "name": "ol8",
  "description": "Martins Oracle Linux 8",
  "versions": [
    {
      "version": "8.4.0",
      "providers": [
        {
          "name": "virtualbox",
          "url": "file:///vagrant/boxes/ol8_8.4.0.box",
          "checksum": "b28a3413d33d4917bc3b8321464c54f22a12dadd612161b36ab20754488f4867",
          "checksum_type": "sha256"
        }
      ]
    },
    {
      "version": "8.4.1",
      "providers": [
        {
          "name": "virtualbox",
          "url": "file:///vagrant/boxes/ol8_8.4.1.box",
          "checksum": "ecb3134d7337a9ae32c303e2dee4fa6e5b9fbbea5a38084097a6b5bde2a56671",
          "checksum_type": "sha256"
        }
      ]
    }
  ]
}

That would be too easy ;) Sadly I don’t have jq available on all the systems I’d like to run this script on. But wait, I have Python available.

Option (2): Python

Although I’m certainly late to to the party I truly enjoy working with Python. Below you’ll find a (shortened) version of a Python script to take care of the metadata addition.

Admittedly it does a few additional things compared to the very basic jq example. For instance, it takes a backup of the metadata file, takes and parses command line arguments etc. It’s a bit longer than a one-liner though ;)

#!/usr/bin/env python3

# PURPOSE
# add metadata about a new box version to the metadata file
# should also work with python2

import json
import argparse
import os
import sys
from time import strftime
import shutil

# Parsing the command line. Use -h to print help
parser = argparse.ArgumentParser()
parser.add_argument("version",       help="the new version of the vagrant box to be added. Must be unique")
parser.add_argument("sha256sum",     help="the sha256 sum of the newly created package.box")
parser.add_argument("box_file",      help="full path to the package.box, eg /vagrant/boxes/ol8_8.4.1.box")
parser.add_argument("metadata_file", help="full path to the metadata file, eg /vagrant/boxes/ol8.json")
args = parser.parse_args()

# this is the JSON element to add
new_box_version = {
    "version": args.version,
    "providers": [
        {
            "name": "virtualbox",
            "url": "file://" + args.box_file,
            "checksum": args.sha256sum,
            "checksum_type": "sha256"
        }
    ]
}

...

# check if the box_file exists
if (not os.path.isfile(args.box_file)):
    sys.exit("FATAL: Vagrant box file {} does not exist".format(args.box_file))

# read the existing metadata file
try:
    with open(args.metadata_file, 'r+') as f:
        metadata = json.load(f)
except OSError as err:
    sys.exit ("FATAL: Cannot open the metadata file {} for reading: {}".format(args.metadata_file, err))

# check if the version to be added exists already. 
all_versions =  metadata["versions"]
if args.version in all_versions.__str__():
    sys.exit ("FATAL: new version {} to be added is a duplicate".format(args.version))

# if the new box doesn't exist already, it's ok to add it
metadata['versions'].append(new_box_version)

# create a backup of the existing file before writing
try:
    bkpfile = args.metadata_file + "_" + strftime("%y%m%d_%H%M%S")
    shutil.copy(args.metadata_file, bkpfile)
except OSError as err:
    sys.exit ("FATAL: cannot create a backup of the metadata file {}".format(err))

# ... and write changes to disk
try:
    with open(args.metadata_file, 'w') as f:
        json.dump(metadata, f, indent=2)
except OSError as err:
    sys.exit ("FATAL: cannot save metadata to {}: {}".format(args.metadata_file, err))

print("INFO: process completed successfully")

That’s it! Next time I need to upgrade my Vagrant boxes I can rely on a fully automated process, saving me quite a bit of time when I’m instantiating a new Vagrant-based environment.