Terraform: create AWS cloud infrastructure for CodeCommit, CodeBuild, ECR services
Preface
All my infrastructure is in AWS cloud and i use Terraform by HashiCorp to create it. This time i will explain, how to create part of CI/CD pipeline in AWS. When i commit my code to repository, build is initiated automatically and docker container is created. Container can be deployed to any server or serverless solution.
AWS CodeCommit is a version control service hosted by Amazon Web Services.
AWS CodeBuild is a fully managed continuous integration service offered by Amazon Web Services (AWS).
AWS EventBridge is a serverless event bus service that makes it easy to connect your applications with data from a variety of sources.
Amazon Elastic Container Registry (Amazon ECR) is an AWS managed container image registry service.
Terraform - infrastructure automation to provision and manage resources in any cloud or data center.
Prerequisites
In order to run this example, following tools has to be installed first:
1. AWS account has to be created and AWS CLI version 2 configured in your computer.
2. Terraform version has to be installed and configured in your computer. In this sample version 1.7.2 is used.
3. I created Node.js "hello world" application and pushed to AWS CodeCommit. We will use it further while working with Terraform. Repository was created separately, it is not managed by Terraform.
As AWS CodeCommit is private repository, I copied the same code to GitHub and it is publicly available, ready for review.
Folder and file layout
- main.tf is main Terraform file. Usually we place cloud provider configuration in it, resources, data sources.
- variables.tf is a place for variables, that can be reused in all resources.
- codebuild.tf resources, related to CodeBuild. Here I would like to mention file buildspec.yml. It has CodeBuild instructions and it is used to build the docker image. And buildspec.yml is not under folder terraform, but inside the root directory of project example-codebuild. And it has to be there, because CodeBuild takes it from here before starting to build the image.
- eventbridge.tf resources, related to AWS EventBridge. This one is a little bit confusing. Some time ago, event rules were part of other AWS service - Cloudwatch. Later AWS moved them to EventBridge, but Terraform didn't rename resource titles and it still works as Cloudwatch. For this reason you can find "cloudwatch" resources inside eventbridge.tf. Everything is fine here, just naming differences between Terraform and AWS.
In this section I mentioned the key points and will not go through each Terraform file line by line. There are detailed description of each resource in Terraform docs, please check while reading *tf files: https://registry.terraform.io/providers/hashicorp/aws/latest/docs
In the next section I will explain how to run what the project and get the result. I will go through each command, step by step.
Create AWS infrastructure
$ git clone ssh://git-codecommit.eu-central-1.amazonaws.com/v1/repos/example-codebuild
Cloning into 'example-codebuild'...
remote: Counting objects: 18, done.
Receiving objects: 100% (18/18), 11.20 KiB | 5.60 MiB/s, done.
Resolving deltas: 100% (4/4), done.
$ cd example-codebuild/
$ ls
Dockerfile app.js buildspec.yml package-lock.json package.json terraform
$ ls
Dockerfile app.js buildspec.yml package-lock.json package.json terraform
$ cd terraform/
$ ls
cloudwatch.tf codebuild.tf main.tf variables.tf
$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/aws...
- Installing hashicorp/aws v5.34.0...
- Installed hashicorp/aws v5.34.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$
$ terraform apply
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
------------------------------------------------
---------some output skipped--------------------
------------------------------------------------
Plan: 10 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_cloudwatch_event_rule.cloudwatch: Creating...
aws_iam_role.codebuild_role: Creating...
aws_iam_role.cloudwatch_role: Creating...
aws_ecr_repository.registry: Creating...
aws_iam_policy.codebuild_policy: Creating...
aws_cloudwatch_event_rule.cloudwatch: Creation complete after 1s [id=example-codebuild]
aws_ecr_repository.registry: Creation complete after 1s [id=example-codebuild]
aws_iam_policy.codebuild_policy: Creation complete after 1s [id=arn:aws:iam::XXXXXXXXXXXX:policy/example-codebuild-policy]
aws_iam_role.codebuild_role: Creation complete after 1s [id=example-codebuild]
aws_iam_role_policy_attachment.codebuild_attachment: Creating...
aws_iam_role.cloudwatch_role: Creation complete after 1s [id=example-codebuild-cloudwatch]
aws_codebuild_project.codebuild: Creating...
aws_iam_role_policy_attachment.codebuild_attachment: Creation complete after 1s [id=example-codebuild-20240201182335522100000001]
aws_codebuild_project.codebuild: Creation complete after 9s [id=arn:aws:codebuild:eu-central-1:XXXXXXXXXXXX:project/example-codebuild]
aws_iam_policy.cloudwatch_policy: Creating...
aws_cloudwatch_event_target.example: Creating...
aws_cloudwatch_event_target.example: Creation complete after 0s [id=example-codebuild-terraform-20240201182344115100000002]
aws_iam_policy.cloudwatch_policy: Creation complete after 1s [id=arn:aws:iam::XXXXXXXXXXXX:policy/example-codebuild-cloudwatch-policy]
aws_iam_role_policy_attachment.cloudwatch_attachment: Creating...
aws_iam_role_policy_attachment.cloudwatch_attachment: Creation complete after 0s [id=example-codebuild-cloudwatch-20240201182344755500000003]
Apply complete! Resources: 10 added, 0 changed, 0 destroyed.
$
Let's have a look at AWS console, we have a new CodeBuild project:
And this is new AWS Elastic Container Registry repository:
Build the application
$ ls
Dockerfile app.js buildspec.yml package-lock.json package.json terraform
$ vim app.js
$ git diff
diff --git a/app.js b/app.js
index f20a6c2..1a38110 100644
--- a/app.js
+++ b/app.js
@@ -3,7 +3,7 @@ const app = express();
const port = 3000;
app.get('/', (req, res) => {
- res.send('Hello World! Greetings!');
+ res.send('Hello World!');
});
app.listen(port, () => {
$
$ git add app.js
$ git commit -m "Small change"
[master 09a643f] Small change
1 file changed, 1 insertion(+), 1 deletion(-)
$ git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 285 bytes | 285.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Validating objects: 100%
To ssh://git-codecommit.eu-central-1.amazonaws.com/v1/repos/example-codebuild
271a164..09a643f master -> master
$
CodeBuild starts to work:
After couple of minutes we have new docker image ready in Amazon Elastic Container Registry
Pull & run docker container locally
$ aws ecr get-login-password --region eu-central-1 | docker login --username AWS --password-stdin XXXXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com
Login Succeeded
$ docker pull XXXXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com/example-codebuild
Using default tag: latest
latest: Pulling from example-codebuild
c57ee5000d61: Pull complete
62907001545e: Pull complete
9aedfdb7ec8c: Pull complete
81915ffe33a8: Pull complete
fcd96bd15e51: Pull complete
a522a29b2395: Pull complete
bef3ffd224f7: Pull complete
20c9f04fa25b: Pull complete
e887fb7272c6: Pull complete
Digest: sha256:db9d11b13012a12730a95431dd6077b40170fccec4baafae810dff9cb8603a4e
Status: Downloaded newer image for XXXXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com/example-codebuild:latest
XXXXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com/example-codebuild:latest
$
$ docker run -p 8080:3000 XXXXXXXXXXXX.dkr.ecr.eu-central-1.amazonaws.com/example-codebuild
Listening at port 3000
This is our application, running in docker container on localhost. Hello world!
After you started "hello world" application, I suggest to play a little but - try to change something. Event some small change like renaming Terraform resource or add a new resource. Build everything again and see how it changes. You will get deeper knowledge how it works.
Finally, this project is just a sample. For production it should be improved, at least remote Terraform state added.
Last Update: Feb 02, 2024 / 02:49 AM
Comments