BLOG: How we use Terraform to setup scenarios in our Autonomous Transport System
Setting up and arranging staging environments in a microservice architecture can be cumbersome. At Autonomous Systems we extended Terraform and created Siteformation to solve this problem.
Test data in staging environments have always been hard to manage. Should you copy data from production? How do you reset the test data? Should every developer have their own set of test data? These are just a few of the questions being raised when trying to tackle the question of test data and configuration.
Background
At Autonomous Transport Systems we develop software both onboard (running onboard the truck) and offboard (running outside the truck). The offboard software can be heavily configured for different autonomous transports and since it built up using a microservice architecture, we have an API-first approach to setting up the data and configuration needed.
Our first approach
We began, just like with most others, to interact with our APIs with tools such as Postman or similar. This approach usually works quite well in the begin when the APIs are few and the system are not that complex. Everyone needs to maintain and keep track of what they create and make sure the created entities reference each other correctly to achieve what you want. However, when achieving what you want takes multiple API calls where information needs to be passed between them it usually starts to become a slow and non-maintainable situation.
Thinking about it differently
So to solve this we sat down and discussed ideas on how we could approach this. We identified that we would like to define the test data and configuration with something that we already were familiar with, code. This is an approach we have adopted for other problems, such as cloud infrastructure with tools such as CloudFormation, Terraform and AWS CDK. These tools allow you to version control, test and apply continuous delivery for cloud infrastructure.
Extending Terraform
Terraform was by far the most versatile and extendable out of the tools we looked in to. It supports developers to define their own Provider. A provider is a plugin for Terraform that defines resources you can manage in basically any system that you can interact with through an API. For example, there are providers for everything from large cloud providers (ex AWS, Azure and GCP) to software tools (ex Grafana, Cassandra and Gitlab). Here is an example of a terraform file defining a VPC in AWS:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 3.0"
}
}
}
# Configure the AWS Provider
provider "aws" {
region = "us-east-1"
}
# Create a VPC
resource "aws_vpc" "example" {
cidr_block = "10.0.0.0/16"
}
A provider is an application built in Go which then Terraform interacts with to manage the defined resources. So we began building our own provider in Go and defined our own resources we would like to manage through Terraform. You are given a structured interface to impement when defining your resources:
func resourceArea() *schema.Resource {
return &schema.Resource{
Create: resourceAreaCreate,
Read: resourceAreaRead,
Update: resourceAreaUpdate,
Delete: resourceAreaDelete,
Schema: map[string]*schema.Schema{
"display_name": {
Type: schema.TypeString,
Required: true,
Description: `The name of the area.`,
},
"area_id": {
Type: schema.TypeString,
Computed: true,
Description: `Id of the area`,
},
"create_timestamp": {
Type: schema.TypeString,
Computed: true,
Description: `Date and time when the area was created`,
},
"update_timestamp": {
Type: schema.TypeString,
Computed: true,
Description: `Date and time when the area was last updated`,
},
},
}
}
You are expected to define the functions for create, read, update and delete. These functions are then responsible for performing the needed actions against the target system, in our case calling APIs in our autonomous offboard system. The expected schema for the resource are also needed for Terraform to keep track of the state so that you at a later point can update and remove your resources in the system. Everything else is handled by Terraform.
The result - Siteformation
After just a few weeks of development we had a first version ready to be released. Our Terraform provider, or Siteformation as we named it, immediately became the go-to tool for managing resources in our autonomous offboard system. It allowed teams to arrange test data and configuration in a more structured way than before. The terraform files became recipes for setting up different scenarios and started to be version-controlled and shared between each other. **A developer could now setup and tear down a complex autonomous operation configuration in our staging environment within seconds using a single command allowing us to iterate our solutions faster than before**.
Siteformation also became an integrated tool within other areas such as simulation and integration testing. Here is an example of how one of these terraform files could look like:
resource "siteformation" "my-area" {
display_name = "My own autonomous operation configuration"
}
resource "siteformation_truck" "my-truck" {
area_id = siteformation_area.my-area.id
display_name = "My own heavy truck"
weight_in_tonnes = 10
sensor_setup {
radars = true
lidars = true
cameras = true
}
}
We clearly see the power in being able to configure our own system (not only infrastructure) using code hence almost all new features in our system are configurable through Siteformation today.
If this sounds interesting consider joining us at Autonomous Systems!
