Benjamin Sago / ogham / cairnrefinery / etc…

Technical notes Manage GitHub repos with Terraform

If you need to configure a large number of GitHub repositories — making sure they all have consistent issue labels and colours, the right features enabled, the correct users added as collaborators, that sort of thing — you’ve probably wished for a better way to configure them all other than making the same repetitive changes using the Web interface over and over again.

It turns out that there is such a tool, though you may have come across it in a completely different context: HashiCorp’s Terraform.

Terraform is most well-known as a network infrastructure automation tool. I use it to create all sorts of infrastructure: the VPSes my sites run on, the DNS entries for my domains, and uptime checks for each site. But in very general terms, Terraform can be thought of as an resource management system, where various resources can be created with certain properties as specified by one or more configuration files. I like to think of it as “a way to turn GUI settings into text settings”.

Here, however, instead of provisioning network infrastructure, the resources will be GitHub repositories, issue tags, and user memberships.

Getting set up

Terraform does not have its service integrations built-in: it requires you to use providers, and in this instance, we are going to use the GitHub provider.

It works by examining all the .tf files inside the current directory, and I like to use providers.tf for putting the providers in.

These files are written in HCL. An example for what to include is also given on the Terraform Registry website, but I’ll reproduce it here:

providers.tf
terraform {
  required_version = ">= 0.13"

  required_providers {
    github = {
      source = "integrations/github"
      version = "4.12.1"  // or whatever the latest is
    }
  }
}

provider github {
  owner = "<your-github-username>"
}

You’ll need a GitHub token in order to authenticate with the API. As the provider documentation states, you can embed this directly within the Terraform file, but like all secrets, it’s best to not hard-code it and load it into the GITHUB_TOKEN environment variable if possible.

With this in place, you can run terraform init to download the provider.

Configuring a repository

Let’s start by setting some basic metadata for a repository: the description, list of tags, the issues and wiki toggles, stuff like that.

We are going to use the github_repository resource to configure our repository. I’m going to use the specsheet repository as an example. Here’s how I have it declared:

specsheet.tf
resource "github_repository" "specsheet" {
  name         = "specsheet"
  description  = "The testing toolkit."
  homepage_url = "https://specsheet.software/"
  topics       = ["command-line", "testing", "rust"]

  has_issues    = true
  has_projects  = false
  has_wiki      = false
  has_downloads = true
}

Now there’s a problem: the repository you are configuring almost certainly already exists, but Terraform, by default, can only configure resources it itself has created, so if you ran terraform apply, it would attempt to create the specsheet repository, receive an error from the GitHub API, and stop and do nothing. (If you really are creating a new repository using Terraform, rather than modifying an existing one, you can skip this step.)

To deal with this, we need to use the terraform import command. Running this command links the repository defined in the Terraform source with the actual repository on GitHub, so that Terraform knows it exists and does not try to create a new one. (The fields do not need to match up — you can import a resource and modify it after it’s been imported.)

This command takes two arguments, the resource name and the repository name, like so:

terraform import github_repository.specsheet specsheet

The first argument, github_repository.specsheet, is the name of the resource in our Terraform file; the second, specsheet, is the name of the repository on GitHub. I don’t see any reason to give them different names, but you still need to know which is which.

Finally, running terraform apply should show you a diff between the “remote resource” and the “local resource” — here, the remote GitHub repository state and the fields we have specified in the file — and prompt you for confirmation before amending the repository settings to match.

Configuring issue tags

Next, let’s do issue tags. For each issue tag we want to create in a repository, we define an instance of an github_issue_label resource.

Two common ones to define are “build error”, which we want highlighted in red, and “feature request”, which we want highlighted in green. If the repository holds the source code of a program, it’s also common to define tags for parts of the program; here, I have an “ui” tag for Specsheet’s UI.

The Terraform to create these three tags looks like this:

specsheet.tf
resource "github_issue_label" "specsheet_build_error" {
  repository  = "specsheet"
  name        = "errors › build error"
  color       = local.error
  description = "specsheet fails to build"
}

resource "github_issue_label" "specsheet_feature_request" {
  repository  = "specsheet"
  name        = "feature request"
  color       = local.request
  description = "New features to add to specsheet"
}

resource "github_issue_label" "specsheet_feature_ui" {
  repository  = "specsheet"
  name        = "features › ui"
  color       = local.feature
  description = "Things to do with the UI"
}

Colours are local variables, which we define in their own locals block. As with anything in Terraform, this can be placed wherever you like, but I tend to put it in colours.tf:

colours.tf
locals {
  error     = "e11d21"
  community = "1d76db"
  request   = "009800"
  feature   = "fbca04"
  help      = "eb6420"
  os        = "bfdadc"
  packaging = "f9d0c4"
  themes    = "ffa2cb"
  notabug   = "cccccc"
}

To run Terraform and create the tags, run terraform apply. Any GitHub tags that already exist with the defined names will be updated to use the new colours, and any tags that didn’t already exist will be created. (Tags that exist but are not specified will not be touched.)

Looping through lists

The last thing we’re going to do is define tags programatically, in a loop.

Specsheet has many checks, and I want to have an issue tag for each check type. These tags all follow the same pattern: a name with a prefix, a description, and a certain colour.

First, I define my list of tags using a locals variable:

specsheet.tf
locals {
  check_types = [
    "apt", "defaults", "fs", "gem", "group", "hash", "homebrew",
    "npm", "systemd", "ufw", "user",
  ]
}

Then, I use the for_each Terraform meta-argument to loop through the list, creating a resource for every check type:

specsheet.tf
resource "github_issue_label" "specsheet_check" {
  for_each = toset(local.check_types)

  repository  = "specsheet"
  name        = "check type › ${each.value}"
  color       = local.themes
  description = "The ‘${each.value}’ check type"
}

Now, all that is needed to add a tag for a new check type is to add a string to an array and run terraform apply.

And in case you were wondering: having the toset function in the for_each field is necessary for fiddly Terraform reasons (that, to be honest, I don’t buy, but it’s not up to me). See the documentation for the explanation.