nodejs-mysql-cloudformation

An example of how you can perform Infrastructure-As-Code (IaC) using AWS CloudFormation and Continuous-Integration/Continuous-Deployment (CI/CD) using AWS CodePipeline.

We will be deploying the NodeJS-ExpressJS-MySQL Create-Read-Update-Delete (CRUD) application at DigiPie/nodejs-mysql-aws-beanstalk, by using the CloudFormation IaC templates in this repository and setting up a simple AWS CodePipeline.

Content

• Architecture
• Getting started
• Continuous-Deployment
• References

Architecture

Figure 1: Architecture diagram created using Lucidchart. Original diagram available here.

This architecture was set up with the following architecture requirements and case study in mind.

VPC stack resources

• VPC
• Public Subnet 1 and 2: Resources inside these subnets are visible to the Internet
• Private Subnet 1 and 2: Resources inside either subnets can only be reached from other resources within the VPC
• Internet Gateway: Used by resources in the Public Subnet 1 and 2 for Internet-access
• NAT Gateway(s): Used as proxies to the Internet by resources in Private Subnet 1 and 2 (one NAT Gateway per AZ for Production)
• Bastion Security Group: Used by the Bastion stack
• Database Security Group: Used by the Database stack
• ELB and App Security Groups: Use by the Elastic Beanstalk stack

Bastion stack resources

• Bastion Host: An EC2 instance in the Public Subnet 1 used for proxying access to resources in Private Subnet 1 and 2
• Cloudwatch Alarms: For detecting invalid SSH attempts to the Bastion

Database stack resources

• Master Database: Master RDS instance used for storing application data (Note: in a real-world scenario, use Amazon Aurora instead for better performance, which is sadly not available in free-tier)
• Read Replica: Read Replica RDS instance used for heavy read operations on the database
• Standby Database: Standby RDS instance in the other Private Subnet/Availability Zone (AZ) for redundancy sake (only for Production)
• Database Subnet Group: For association with VPC Private Subnet 1 and 2

Elastic Beanstalk stack resources

• Elastic Beanstalk Application: Pulls application from S3 bucket for initialization, and from GitHub via CodePipeline subsequently
• Elastic Beanstalk Environment (can create an environment for Development and another for Production)
• Auto-Scaling Group(s): Automatic scaling based on CPU utilization (2 ASGs in both AZ for Production)
• Elastic Load Balancer: Only allows HTTP traffic; can be configured to be Network Load Balancer (Layer-4) or Application Load Balancer (Layer-7)

Other services

These services are not set up by the CloudFormation templates:

• S3 Bucket:
  • Used to store the initial app to be deployed to Elastic Beanstalk when the CloudFormation stack is built
  • Used to store reports generated by the Batch Job
• CodePipeline Pipeline: Used to continuously deploy the updated app from a GitHub repository to Elastic Beanstalk
• Batch Job: Used for daily generation of reports which incur heavy read operations; reads from the Read Replica to mitigate impact on Master Database and the Web Application

Getting started

Pre-requisites:

• EC2 key pair
• npm
• git
• MySQL Workbench (optional)

Setup VPC, Bastion and Database stacks

1. Clone this repository to your local disk.
2. Open the AWS Cloudformation console.
3. Click Create stack > With new resources (standard).
4. Select Prepare template: 'Template is ready'.
5. Select Template source: 'Upload a template file'.
6. Before clicking Create stack at the final step for any stack you create, make sure you select Capabilities: 'I acknowledge that AWS CloudFormation might create IAM resources' if it appears. Otherwise your stack deployment will fail.
7. Launch the vpc.cfn.yml stack first and fill in the parameters accordingly. Keep track of what you fill as some of these values will be referenced later on.
8. After the VPC stack has a 'CREATE_COMPLETE' status, launch the bastion.cfn.yml and db.cfn.yml stacks. You can do so concurrently given the Bastion and Database stacks do not rely on each other; they only rely on the VPC stack.

Setup Database table

1. Connect to the main RDS database via the bastion host with a database client such as MySQL Workbench.

   • You cannot connect to the main and read replica databases directly given they are in the VPC's private subnets. You need to connect through the bastion host in the public subnet.
   • See the Connecting to Your Instances and Database section of AWS: Building a VPC with the AWS Startup Kit.
   • You can find the required parameters for your database connection in the AWS Cloudformation console > Select your Database stack > Outputs.

2. After connecting to the main RDS database, execute the following SQL script to setup the table used by the Web Application:

   CREATE DATABASE test;
   
   USE test;
   
   CREATE TABLE users (
     id int(11) NOT NULL auto_increment,
     name varchar(100) NOT NULL,
     age int(3) NOT NULL,
     email varchar(100) NOT NULL,
     PRIMARY KEY (id)
   );

Prepare your app for deployment

1. Clone the app repository DigiPie/nodejs-mysql-aws-beanstalk to your local disk.

2. In the app's root directory, zip the code by running the following npm command:

   npm run zip

3. Open the AWS S3 Bucket console.

4. Click Create bucket.

5. Click Create bucket using all default settings.

6. Upload the created zip file on the bucket.

7. Note down the bucket and zip file name (e.g. 'my-bucket-evan' and 'nodejs-mysql-crud.zip' respectively). You will use them as parameters for your Elastic Beanstalk stack.

Setup Elastic Beanstalk stack

1. Return to the AWS Cloudformation console.

2. Launch the elastic-beanstalk.cfn.yml stack and fill in parameters as follows:

   • AppS3Bucket: Name of the S3 bucket that contains the app zip file (e.g. 'my-bucket-evan')
   • AppS3Key: Name of the app zip file (e.g. 'nodejs-mysql-crud.zip')
   • NetworkStackName: Name of your VPC stack
   • DatabaseStackName: Name of your DB stack
   • SSLCertificateArn: Optional, leave it empty for this walkthrough

3. Before launching your stack, select Capabilities: 'I acknowledge that AWS CloudFormation might create IAM resources'.

4. After your Elastic Beanstalk stack has 'CREATE_COMPLETE' status, visit your application at Output > EnvironmentURL.

You are done!

Continuous Deployment

To continuously deploy your application from GitHub to Elastic Beanstalk automatically on push to master, you can use either AWS CodePipeline or GitHub Action. In this case, we will be using AWS CodePipeline. For GitHub Action, refer to DigiPie/nodejs-mysql-aws-beanstalk for more instructions.

1. Open the AWS CodePipeline console.
2. Click Create pipeline.
3. Name your Pipeline and use the auto-filled Role name (e.g. 'AWSCodePipelineServiceRole-ap-southeast-1-test').
4. Select Source provider: 'GitHub (Version 1)'.
5. Click Connect to GitHub.
6. Select the Repository and Branch.
7. Use the default detection option: 'GitHub webhooks (recommended)'.
8. Click Skip build stage.
9. Select Deploy provider: 'AWS Elastic Beanstalk'.
10. Choose the appropriate Region, Application name and Environment name.
11. Click Create pipeline.
12. Test your pipeline by pushing a commit to master.

References

This repository was only possible thanks to the following resources:

• Sample application: Node.js, Express & MySQL: Simple Add, Edit, Delete, View (CRUD)
• Sample AWS Elastic Beanstalk project: amazon-archives/startup-kit-nodejs
• Sample AWS CloudFormation templates: aws-samples/startup-kit-templates
• Continuous Deployment pipeline: AWS: Set up a Continuous Deployment Pipeline