Shing Lyu

Disclaimer: The views expressed here are my own; they do not reflect the views of my current and past employers.

Moving AWS Service across accounts using Terraform

Recently I was assigned the task to move our REST API hosted on AWS to a new account. The organization I worked for is moving to a one-account-per-team (or a few closely related teams) approach, as opposed to one account shared by all teams. Having one account per team helps reduce the clutter in the accounts because you only see your resources. It also helps the platform/SRE team to control the cost in a more fine-grain manner. Since we have everything in AWS, it also reduces the chance that we hit AWS resource limits.

The service we built is all provisioned using Terraform. They are tested and deployed with Drone CI tool. I’ll discuss key points to considerations when migrating accounts across accounts.

(continue reading...)

Switching Between Multiple Local Backends in Terraform

Terraform has many backend types. The local backend stores the state on the local filesystem, so it’s ideal for quick local testing. By it’s not very obvious how to have multiple local backend and state, and how to easily switch between them. One use case for this is when you deploy the same set of resources to multiple AWS regions. Let’s say we want to create two API gateways and their corresponding DNS records to two regions. We use the aws_route53_record resource to deploy them:

resource "aws_route53_record" "api" {
  name    = "${var.api_url}"
  type    = "A"
  # the rest are omitted

And we want to set var.api_url to and for Europe and US regions in two separate tfvars file.

Then if you try to apply them sequentially like so:

terraform init
terraform apply -var-file=eu.tfvars
terraform apply -var-file=us.tfvars

You’ll notice that the second apply will try to destroy your record, and replace it with an record. This is because the states are the same, and the resource name is the same between two apply attempts, so terraform think you want to destroy the existing record and create a new one. There is also a problem when you try to destroy resources. Because the resources have the same name, so if you destroy them in one region, you won’t be able to destroy then in the other one. Because terraform assumes everything is already gone.

(continue reading...)

Simplify Your CI Pipeline Configuration with Jsonnet

This post is also featured on the DAZN Engineering Blog.

Most of the CI/CD (Continuous Integration/Continuous Delivery) tools nowadays supports some form of configuration file so you can properly version control them. For example Travis CI, Gitlab CI, Circle CI and Drone CI uses YAML file. Jenkins uses its own DSL. These YAML-based configuration files are easy to read and edit, but they don’t scale very well when the file grows big. This problem can be solved by using a nice data templating language called Jsonnet. In this post we’ll be demonstrating Drone CI v1.0 configuration file format, but the idea can be easily applied to other CI tool.

(continue reading...)

Download JavaScript Data as Files on the Client Side

When building websites or web apps, creating a “Download as file” link is quite useful. For example if you want to allow user to export some data as JSON, CSV or plain text files so they can open them in external programs or load them back later. Usually this requires a web server to format the file and serve it. But actually you can export arbitrary JavaScript variable to file entirely on the client side. I have implemented that function in one of my project, MozApoy, and here I’ll explain how I did that.

(continue reading...)

AWS Route 53 Load Balancing with Terraform

Terraform has some great documentation on Route 53, but it’s a little bit hard to understand how all the resources works together. So to demonstrate, we are going to build an REST API that is deployed to multiple AWS regions, which has one public-facing URL, which is load balanced through Route 53. There are some additional requirements:

(continue reading...)