This Website

My Development Pipeline

Note: this has been deprecated and is no longer active since the switch to Terraform from Cloudformation. It was a fun exercise to get set up, but not worth the effort to migrate.

Tools

Goal

I wanted this project to have a somewhat sustainable and reasonably quick development pipeline. I wanted most processes to be automated after a commit so I could spend most of my time working on the website and less of my time figuring out how to deploy environments to test out my changes.

My desired developer workflow was something of the following:

  1. Check out repo
  2. Make feature branch off of develop branch
  3. Make changes, test locally, and push changes
  4. Test changes on automatically deployed test environment
  5. Open PR, merge feature branch into develop
  6. Test changes on automatically deployed stage environment
  7. Open PR, merge develop into master
  8. Changes automatically deployed to live

This workflow is obviously over the top for a project with a single developer but I thought it'd be a fun learning experience to build.

This is what I did

I first set up three environments, `live`, `stage`, and `test`. The `test` environment ended up being a little bit different, but I'll get to this later. For each environment, I made an S3 bucket. I wrote a quick `index.html`, uploaded, and I was live!

I was planning on setting up most of the AWS infrastructure and eventual backend through AWS Cloudformation. I chose to use Cloudformation mostly because I don't have too much experience with Cloudformation (as opposed to Terraform, which is wonderful). Through Travis, I set up S3 deploys to `live`, `stage`, and `test` based on the branch name, `master` goes to `live`, `develop` to `stage` and everything else to `test`. On top of this, I set up a Cloudformation template that would deploy a simple serverless stack pointing to a stubbed Lambda function. At this point, each commit would do the following:

  1. Create a Cloudformation stack with a unique name with a Lambda function and associated API Gateway
  2. Upload the static website files to the S3 bucket associated with the given branch

This wasn't enough. I didn't like how if I had multiple branches for features at the same time, only one would be able to exist in my `test` bucket at a time. This wasn't _actually_ an issue since, again, I'm the only one working on the website, but it sounded like a fun problem to solve.

To solve this, I updated my Travis script to deploy feature branches to a location in S3 prefixed with the branch name. For example, pushing `index.html` in branch called `my-branch` would be uploaded to `test-bucket/sandboxes/my-branch/index.html` instead of `test-bucket/index.html` as it was formally. This enabled the existence of multiple copies of the website at different URL paths.

I then set up a basic frontend for the homepage of the test site. On load, this page listed all the keys in the root of the `test` S3 bucket and provided links to all of the available paths. My own little testing hub with updated environments available on every push -- sweet.

Add some deletion of the old stacks that correspond to the environment and branch that is being deployed, add some lifecycle rules to the buckets that host the static files, and we have a sustainable development pipeline!

Go back home