Creating infrastructure with code is an extremely powerful tool that can abstract away the need for on-premises hardware. Instead of attempting to hand provision things in private cloud consoles such as AWS, GCP, or Azure we can instead rely on tools such as Terraform and Pulumi to help automate this and in both cases help to keep track of the state of the infrastructure. Sometimes, if you just need something quickly, one could also use Bash and the aws-cli to provision the same. Be aware, however, that you would need to write an accompanying uninstall script and/or destroy the resources manually.
After trying out both Terraform and Pulumi it’s safe to say that Pulumi is extremely easy to use, fast, and developer friendly. For the purposes of a demo let’s install the cli tool with brew install pulumi for those on macOS. Next, we can run pulumi new <project-name> and the CLI will walk you through provisioning a new project. For the purposes of comparison let’s try and serve hello world at port 80 on an ec2 instance in AWS. Using the pulumi python library that will look something like the following:
#!/usr/bin/env python3
from pulumi import export
import pulumi_aws as aws
vpc = aws.ec2.Vpc(
"ec2-vpc",
cidr_block="10.0.0.0/16"
)
public_subnet = aws.ec2.Subnet(
"ec2-public-subnet",
cidr_block="10.0.101.0/24",
tags={
"Name": "ec2-public"
},
vpc_id=vpc.id
)
igw = aws.ec2.InternetGateway(
"ec2-igw",
vpc_id=vpc.id,
)
route_table = aws.ec2.RouteTable(
"ec2-route-table",
vpc_id=vpc.id,
routes=[
{
"cidr_block": "0.0.0.0/0",
"gateway_id": igw.id
}
]
)
rt_assoc = aws.ec2.RouteTableAssociation(
"ec2-rta",
route_table_id=route_table.id,
subnet_id=public_subnet.id
)
sg = aws.ec2.SecurityGroup(
"ec2-http-sg",
description="Allow HTTP traffic to EC2 instance",
ingress=[{
"protocol": "tcp",
"from_port": 80,
"to_port": 80,
"cidr_blocks": ["0.0.0.0/0"],
}],
vpc_id=vpc.id,
)
ami = aws.ec2.get_ami(
most_recent="true",
owners=["amazon"],
filters=[{"name": "name", "values": ["amzn-ami-hvm-*"]}]
)
user_data = """
#!/bin/bash
echo "Hello, world!" > index.html
nohup python -m SimpleHTTPServer 80 &
"""
ec2_instance = aws.ec2.Instance(
"ec2-tutorial",
instance_type="t2.micro",
vpc_security_group_ids=[sg.id],
ami=ami.id,
user_data=user_data,
subnet_id=public_subnet.id,
associate_public_ip_address=True,
)
export("ec2-public-ip", ec2_instance.public_ip)
To run the configuration you can simply issue pulumi up from the root directory of the pulumi project and the script will provision the specified resources and save the state of the resources! Run time was about 20 seconds to provision the resources. We can verify the installation went ok by using curl to contact our simple HTTP server with curl <public_ip> and to tear down the resources use pulumi destroy and the specified resources will be torn down, this took about 1 minute.
Similarly, to do the same in Terraform is a little bit more leg work on behalf of the module creator. Whereas in Pulumi we didn’t need to be quite so explicit, Terraform will expect extremely explicit and well planned inputs. If you are on macOS install Hashicorp’s Terraform with:
- brew install hashicorp/tap/terraform
- brew install terraform
A helpful alias is
alias tf=terraform, so you can skip some typing later on. We’re going to lay out the Terraform project something like:
tree .
.
├── ec2
│ ├── main.tf
│ └── outputs.tf
├── main.tf
└── outputs.tf
We’ll create an ec2 module, best practice might dictate factoring some of the components like the vpc to it’s own module, but in the interest of brevity let’s call it a complete ec2 example including everything the ec2 instance might need:
main.tf
terraform {
required_version = ">= 0.13.1"
required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.7"
}
}
}
module "ec2-terraform" {
source = "./ec2"
}
outputs.tf
- We need to pass the public ip of the ec2 up from the module
output "ec2_module_says" {
value = module.ec2-terraform.public_ip
}
ec2/main.tf
provider "aws" {
region = local.region
}
locals {
name = "example-ec2"
region = "us-west-2"
main_vpc_cidr = "10.0.0.0/24"
public_subnets = "10.0.0.128/26"
private_subnets = "10.0.0.192/26"
user_data = <<-EOL
#!/bin/bash -xe
echo "Hello Terraform!" > index.html
nohup python -m SimpleHTTPServer 80 &
EOL
tags = {
Owner = "user"
Environment = "dev"
}
}
resource "aws_vpc" "Main" {
cidr_block = local.main_vpc_cidr
instance_tenancy = "default"
}
resource "aws_internet_gateway" "IGW" {
vpc_id = aws_vpc.Main.id
}
resource "aws_subnet" "publicsubnets" {
vpc_id = aws_vpc.Main.id
cidr_block = local.public_subnets
}
resource "aws_subnet" "privatesubnets" {
vpc_id = aws_vpc.Main.id
cidr_block = local.private_subnets
}
resource "aws_route_table" "PublicRT" {
vpc_id = aws_vpc.Main.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.IGW.id
}
}
resource "aws_route_table" "PrivateRT" {
vpc_id = aws_vpc.Main.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.NATgw.id
}
}
resource "aws_route_table_association" "PublicRTassociation" {
subnet_id = aws_subnet.publicsubnets.id
route_table_id = aws_route_table.PublicRT.id
}
resource "aws_route_table_association" "PrivateRTassociation" {
subnet_id = aws_subnet.privatesubnets.id
route_table_id = aws_route_table.PrivateRT.id
}
resource "aws_eip" "nateIP" {
vpc = true
}
resource "aws_nat_gateway" "NATgw" {
allocation_id = aws_eip.nateIP.id
subnet_id = aws_subnet.publicsubnets.id
}
resource "aws_instance" "example_ec2_host" {
count = 1
instance_type = "t3.micro"
subnet_id = aws_subnet.publicsubnets.id
ami = data.aws_ami.amazon_linux.id
associate_public_ip_address = true
vpc_security_group_ids = [aws_security_group.example_sg_ec2.id]
user_data = local.user_data
tags = {
Name = "ec2-example"
Terraform = "true"
}
}
# Create the security groups
resource "aws_security_group" "example_sg_ec2" {
vpc_id = aws_vpc.Main.id
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow ssh from any"
}
ingress {
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow http from any"
}
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow https from any"
}
ingress {
cidr_blocks = ["0.0.0.0/0"]
from_port = 8
to_port = 0
protocol = "icmp"
description = "Allow ping from any"
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
description = "Allow out to any"
}
}
data "aws_ami" "amazon_linux" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["amzn-ami-hvm-*-x86_64-gp2"]
}
}
ec2/outputs.tf
output "public_ip" {
description = "The public IP address assigned to the instance"
value = try(aws_instance.example_ec2_host[0].public_ip, "")
}
Once these files are defined and from the root directory of the project run tf init to initialize the Terraform project. Next, let’s plan the project and see what will be provisioned with tf plan -out=out.tfplan, and finally, if the changes are satisfactory, apply the plan with tf apply "out.tfplan". Resource creation took about a minute and twenty seconds, but I’ve seen it hang when provisioning resources in AWS so be patient. Once we’ve verified the HTTP server with curl <public_ip> we can tear everything down with a tf destroy.
For completeness checkout the following gist of an aws-cli script provisioning the same resources. As mentioned at the outset managing this installation and its configuration could be challenging to do manually which is something both Pulumi and Terraform do out of the box!
References
- https://www.learnaws.org/2021/06/19/pulumi-python-ec2/
- https://dev.to/mkabumattar/how-to-create-an-aws-ec2-instance-using-aws-cli-32ek
- https://www.agix.com.au/terraform-creating-a-vpc-with-an-ec2/