Private Fargate Deployment with VPC Endpoints

VPC Endpoints allow you to have private containers, pulled from ECR repositories, with no external network ingress or egress. These containers can enhance your infrastructure security posture by utilizing AWS PrivateLink, restricting packets from going over the public internet.

Assumptions/Prereqs

  • You have a private ECR repository, with a tagged image created in your AWS Region.
  • The example code definitions will contain statically coded capacity metrics, as a low-cost deployment. If you are using this in your environment, it is recommended you variablize, or increase the static values.
  • The code will not restrict the endpoints, as it is expected the roles on your resources will preform that restriction, however, there will be linked AWS reference documentation, if you require.
  • The architecture will have zero public subnets or internet egress. You may require this additional infrastructure, but is outside the scope, and shouldn't impact the defined code.

Architecture

The environment will use a trimmed-down version of a common VPC configuration, with only 2 private subnets, and a Fargate instance.

Architecture Diagram

Process

To enable containers to pull from ECR several steps are needed. Below broken down terraform code will walk through an end to end build, but this is the process that code will perform.

  1. Configure a VPC with 2 private subnets and allow DNS Hostname Resolve and Support

  2. Configure a VPC Endpoint Security Group to allow your VPC's CIDR as Ingress

    • This will be utilized for all internal addresses to communicate with the interfaces, as private DNS will resolve to an internal address.

  3. Create an ECR DKR Service Endpoint

  4. Create a S3 Endpoint, which will connect to the starport bucket

    • Service types: com.amazonaws.<region>.s3
    • Note from AWS Documentation:

    The gateway endpoint is required because Amazon ECR uses Amazon S3 to store Docker image layers... The gateway endpoint is required because Amazon ECR uses Amazon S3 to store Docker image layers:

    arn:aws:s3:::prod-<region>-starport-layer-bucket/*

  5. Create Logs endpoint to allow your container to log to CloudWatch

    • Service types: com.amazonaws.<region>.logs

  6. Configure your Fargate service with egress to the VPC and S3 private gateway

Terraform

Variable Declaration

Create a variables.tf file for the ECR Image you will pass in.

variable "app_image" {}
view raw variables.tf hosted with ❤ by GitHub

Provider and Account Data

Initialize the config with providers and data.

provider "aws" {
region = "us-east-1"
}
data "aws_region" "current" {}
data "aws_caller_identity" "current" {}
view raw main.tf hosted with ❤ by GitHub

Security Groups

Provision the Security Groups for the VPC Endpoints and Fargate instance.

resource "aws_security_group" "vpce" {
name = "vpce"
vpc_id = aws_vpc.main.id
ingress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = [aws_vpc.main.cidr_block]
}
tags = {
Environment = "dev"
}
}
resource "aws_security_group" "ecs_task" {
name = "ecs"
vpc_id = aws_vpc.main.id
egress {
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = [aws_vpc.main.cidr_block]
}
egress {
from_port = 443
to_port = 443
protocol = "tcp"
prefix_list_ids = [aws_vpc_endpoint.s3.prefix_list_id]
}
tags = {
Environment = "dev"
}
}
view raw sg.tf hosted with ❤ by GitHub

VPC Endpoints

Configure the VPC Endpoints for S3, DKR, and logging.

resource "aws_vpc_endpoint" "s3" {
vpc_id = aws_vpc.main.id
service_name = "com.amazonaws.${data.aws_region.current.name}.s3"
vpc_endpoint_type = "Gateway"
route_table_ids = [aws_route_table.private.id]
tags = {
Name = "s3-endpoint"
Environment = "dev"
}
}
resource "aws_vpc_endpoint" "dkr" {
vpc_id = aws_vpc.main.id
private_dns_enabled = true
service_name = "com.amazonaws.${data.aws_region.current.name}.ecr.dkr"
vpc_endpoint_type = "Interface"
security_group_ids = [
aws_security_group.vpce.id,
]
subnet_ids = [aws_subnet.private_a.id, aws_subnet.private_b.id]
tags = {
Name = "dkr-endpoint"
Environment = "dev"
}
}
resource "aws_vpc_endpoint" "logs" {
vpc_id = aws_vpc.main.id
private_dns_enabled = true
service_name = "com.amazonaws.${data.aws_region.current.name}.logs"
vpc_endpoint_type = "Interface"
security_group_ids = [
aws_security_group.vpce.id,
]
subnet_ids = [aws_subnet.private_a.id, aws_subnet.private_b.id]
tags = {
Name = "logs-endpoint"
Environment = "dev"
}
}
view raw vpce.tf hosted with ❤ by GitHub

Fargate Service

Configure your IAM Role, task definition, and Fargate service.

data "aws_iam_policy_document" "fargate-role-policy" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = ["ecs.amazonaws.com", "ecs-tasks.amazonaws.com"]
}
}
}
resource "aws_iam_policy" "fargate_execution" {
name = "fargate_execution_policy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability"
],
"Resource": "arn:aws:ecr:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:repository/${var.app_image}"
},
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream"
],
"Resource": "*"
}
]
}
EOF
}
resource "aws_iam_policy" "fargate_task" {
name = "fargate_task_policy"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]
}
EOF
}
resource "aws_iam_role" "fargate_execution" {
name = "fargate_execution_role"
assume_role_policy = data.aws_iam_policy_document.fargate-role-policy.json
}
resource "aws_iam_role" "fargate_task" {
name = "fargate_task_role"
assume_role_policy = data.aws_iam_policy_document.fargate-role-policy.json
}
resource "aws_iam_role_policy_attachment" "fargate-execution" {
role = aws_iam_role.fargate_execution.name
policy_arn = aws_iam_policy.fargate_execution.arn
}
resource "aws_iam_role_policy_attachment" "fargate-task" {
role = aws_iam_role.fargate_task.name
policy_arn = aws_iam_policy.fargate_task.arn
}
# Fargate Container
resource "aws_cloudwatch_log_group" "app" {
name = "/ecs/app"
tags = {
Environment = "dev"
}
}
resource "aws_ecs_cluster" "main" {
depends_on = [aws_cloudwatch_log_group.app]
name = "cluster"
}
locals {
container_defintion = [{
cpu = 1024
image = "${data.aws_caller_identity.current.account_id}.dkr.ecr.${data.aws_region.current.name}.amazonaws.com/${var.app_image}"
memory = 2048
name = "app"
networkMode = "awsvpc"
logConfiguration = {
logdriver = "awslogs"
options = {
"awslogs-group" = "/ecs/app"
"awslogs-region" = data.aws_region.current.name
"awslogs-stream-prefix" = "stdout"
}
}
}]
}
resource "aws_ecs_task_definition" "app" {
family = "app"
network_mode = "awsvpc"
cpu = local.container_defintion.0.cpu
memory = local.container_defintion.0.memory
requires_compatibilities = ["FARGATE"]
container_definitions = jsonencode(local.container_defintion)
execution_role_arn = aws_iam_role.fargate_execution.arn
task_role_arn = aws_iam_role.fargate_task.arn
tags = {
Name = "app"
Environment = "dev"
}
}
resource "aws_ecs_service" "main" {
name = "service"
cluster = aws_ecs_cluster.main.id
task_definition = aws_ecs_task_definition.app.arn
desired_count = "1"
launch_type = "FARGATE"
network_configuration {
security_groups = [aws_security_group.ecs_task.id]
subnets = [aws_subnet.private_a.id]
}
}
view raw fargate.tf hosted with ❤ by GitHub

Execution

terraform apply -var app_image=<your ECR image name>

References

Amazon ECR Interface VPC Endpoints

Updates

(August 2) This post was written with Fargate 1.3. If having issues pulling from registry, ensure to check out this Stack Overflow solution.