Shing Lyu

Lessons learned in writing my first book

You might have noticed that I didn’t update this blog frequently in the past year. It’s not because I’m lazy, but I focused all my creative energy on writing this book: Practical Rust Projects. The book is now available on Apress, Amazon and O’Reilly. In this post, I’ll share some of the lessons I learned in writing this book.

Update AWS Security Groups with Terraform

In theory, Terraform is capable of figuring out the dependency between AWS resources and make updates in the correct order. However, AWS security groups often become a source of trouble if you don’t understand how Terraform handles it. If you are having issues modifying the security group because they are used by other resources, here are some ways you can mitigate that.

Moving AWS Service across accounts using Terraform

Recently I was assigned the task to move our REST API hosted on AWS to a new account. The organization I worked for is moving to a one-account-per-team (or a few closely related teams) approach, as opposed to one account shared by all teams. Having one account per team helps reduce the clutter in the accounts because you only see your resources. It also helps the platform/SRE team to control the cost in a more fine-grain manner. Since we have everything in AWS, it also reduces the chance that we hit AWS resource limits.

The service we built is all provisioned using Terraform. They are tested and deployed with Drone CI tool. I’ll discuss key points to considerations when migrating accounts across accounts.

Switching Between Multiple Local Backends in Terraform

Terraform has many backend types. The local backend stores the state on the local filesystem, so it’s ideal for quick local testing. By it’s not very obvious how to have multiple local backend and state, and how to easily switch between them. One use case for this is when you deploy the same set of resources to multiple AWS regions. Let’s say we want to create two API gateways and their corresponding DNS records to two regions. We use the aws_route53_record resource to deploy them:

resource "aws_route53_record" "api" {
  name    = "${var.api_url}"
  type    = "A"
  # the rest are omitted
}

And we want to set var.api_url to api-eu.example.com and api-us.example.com for Europe and US regions in two separate tfvars file.

Then if you try to apply them sequentially like so:

terraform init
terraform apply -var-file=eu.tfvars
terraform apply -var-file=us.tfvars

You’ll notice that the second apply will try to destroy your api-eu.example.com record, and replace it with an api-us-example.com record. This is because the states are the same, and the resource name is the same between two apply attempts, so terraform think you want to destroy the existing record and create a new one. There is also a problem when you try to destroy resources. Because the resources have the same name, so if you destroy them in one region, you won’t be able to destroy then in the other one. Because terraform assumes everything is already gone.

Simplify Your CI Pipeline Configuration with Jsonnet

This post is also featured on the DAZN Engineering Blog.

Most of the CI/CD (Continuous Integration/Continuous Delivery) tools nowadays supports some form of configuration file so you can properly version control them. For example Travis CI, Gitlab CI, Circle CI and Drone CI uses YAML file. Jenkins uses its own DSL. These YAML-based configuration files are easy to read and edit, but they don’t scale very well when the file grows big. This problem can be solved by using a nice data templating language called Jsonnet. In this post we’ll be demonstrating Drone CI v1.0 configuration file format, but the idea can be easily applied to other CI tool.