m Moritz Bauer — writing on digital data
◦ Essay ·

Terraform and SGTM on Google Cloud Platform

A Terraform script that automates a complete serverside Google Tag Manager setup on GCP — Cloud Run services, log filters, alerts, and auto-updates.

Why bother?

Deploying a serverside Google Tag Manager (SGTM) isn’t complicated, right? In principle it’s quick, unless you want a few extra things — set up a service account, grant IAM permissions, filter logs, deploy an auto-update Cloud Function, and add a few uptime checks plus an alert policy.

So we don’t have to do all this by hand every time we set up a new instance, we can use Terraform. Terraform lets us write what is essentially a recipe — using the “Infrastructure as Code” approach — describing which components we want to deploy where and how.

Since I set up new SGTMs fairly often, I wrote a Terraform script that automatically provisions the following components on Google Cloud Platform (GCP):

  • Enable required APIs: IAM, Run, Cloud Scheduler, Cloud Build, Cloud Functions, Monitoring
  • Create a service account with the required roles: run.invoker, cloudfunctions.invoker, cloudfunctions.serviceAgent
  • 2 SGTM Cloud Run services: Preview + Production
  • Logs Exclusion: exclude default logs for Cloud Run
  • Uptime Checks & Alert Policy: email notifications when the production SGTM is not reachable
  • Auto Updates: my SGTM update Cloud Function automated with Cloud Scheduler (8:00 UTC every day)

How does it work?

First, you need to make sure Terraform and the gcloud CLI are installed. Once that’s done, just clone the GitHub repository.

Authentication with GCP

For the deployment we need a way to authenticate against the Google Cloud Platform. Often I see service account keys being downloaded as JSON files for this. I’d advise against that — if such a key falls into the wrong hands, that person has access to the service account and can use it accordingly.

A safer and, in my view, simpler option is to use Application Default Credentials, or ADC for short. Here, your personal Google account is used for authentication.

gcloud auth application-default login

A Google login window opens, where you identify yourself. The prerequisite for using this method is, of course, that your account already has the necessary permissions in the GCP project where you want to deploy resources.

There are different approaches to permissions. Google recommends the “principle of least privilege”, i.e. granting only as few permissions as necessary. For one-off setup tasks I often use the primitive roles, e.g. the Editor role, or in rare cases the Owner role.

How do I use the Terraform script?

Preparing the deployment / adjusting variables

The setup of my Terraform script is fairly simple. The first thing you have to do is fill in the variables in terraform.tfvars. They control which GCP project and which region the resources are deployed to. For a minimal setup, the variables project_id, container_config, notification_user, and google_storage_bucket_name need to be adjusted.

# Project ID where terraform will build the assests in
project_id = "YOUR_GCP_RPOJECT_ID"

# All resources will be built in this region
region =  "europe-west3"

# Container Config string from GTM Webinterface
container_config = "SGTM_CONTAINER_CONFIG"

# The names for the SGTM Cloud Run services
service_name_preview = "sgtm-preview"
service_name_production = "sgtm-production"

# Controls the maximum and minimum instances for Cloud Run service running the production SGTM
min_instance_count = 0
max_instance_count = 1
cpu_boost = true

# DONT Change this - Log filter to trigger alerts when SGTM got updated
cloud_function_update_filter = "resource.type=\"cloud_function\" \nAND textPayload=~\"Versions are different: Deploying a new revision\""

# Update interval for the Cloud Function updating the SGTM Image in unix-cron job format - Default everyday at 8:00 am UTC
update_interval = "0 8 * * *"

# The filter to define which logs will be excluded
cloud_run_exclusion_filter = "resource.type=\"cloud_run_revision\" AND severity = \"INFO\" OR severity =\"DEFAULT\""

# Used for error notfication alerting
notification_user = {
    name: "YOUR_NAME"
    email: "YOUR_EMAIL"
}

# Used to name the google storage bucket
google_storage_bucket_name = "RANDOM_STRING"

The variables for the Cloud Run exclusion filter and the log filter for the SGTM update notification should not be changed.

Deploy infrastructure

Once everything is prepared, we initialize Terraform. This is done in the terminal of your choice — for me that’s usually the integrated terminal in VS Code.

terraform init

This downloads the providers our script needs. In our case that’s the google provider, which provides the necessary resources.

Next, we can plan our deployment. We could do this explicitly with terraform plan, but applying the script will also show us the planned resources before performing the change.

terraform apply

This gives us an overview of all components the Terraform script will deploy, provided we agree with “yes”.

Once we have agreed, the resources are built up in the correct order. This is enforced through depends_on on the various resources.

In the terminal we can follow the progress of the different steps. If everything succeeds, we get a confirmation of how many resources were set up.

You can of course make manual changes to the components afterwards — e.g. adjust the maximum number of instances in Cloud Run, or add another notification channel like Slack to the alert.

Closing words

Since GCP has gained a more prominent place in the digital analytics space, software engineering and cloud engineering approaches have become increasingly important for our work. Alongside the Google Analytics 4 raw data in BigQuery, the serverside Google Tag Manager is at home on GCP. That doesn’t just open up new options — it also forces us to look beyond the usual horizon every once in a while.

Terraform, combined with GitHub, gives us a way to deploy reliably reproducible setups. Especially for steps that are repeated often, a Terraform script is a good fit. We can ensure no components get forgotten or set up incorrectly. Once defined, the script’s output is predictable.

Another advantage is that Terraform remembers which components were deployed in the last run. This is stored in the terraform.tfstate file, which is created locally. So when developing the script further, we can simply pull the latest version from GitHub and apply only the changes to our project — also via terraform apply.

Instead of using the Terraform script locally, we could also automate it with GitHub Actions. An automation could be set up where project resources are automatically updated when there’s a new commit on GitHub.