A simple image CI/CD pipeline using combination of Github + Jenkins + Hashicorp Packer + Chef + serverspec. Resources are built using Hashicorp Terraform targetting AWS.
Clone folder cookbooks of this repo and set up appropriate vars in cookbooks/jenkins-wrapper/attributes/default.rb file. Specific variable explanations are below:
| Param | Explanation |
|---|---|
| ['jenkins-wrapper']['github']['user'] | Your Github role userid whose token was put into github_token in S3 bucket |
| ['jenkins-wrapper']['github']['admins'] | Privileged user(s) that can authorize PR builds and promotions |
| ['jenkins-wrapper']['github']['repo'] | Github repo to trigger this pipeline |
| ['jenkins-wrapper']['github']['credential_id'] | Random string generated by uuidgen, to be used when creating Jenkins credential |
| 'jenkins-wrapper']['github']['github_auth_id'] | Random string generated by uuidgen, to be used when creating Jenkins credential |
Upload all cookbooks in cookbooks folder to your Chef server, along with all dependency cookbooks (you can use berks utility to do this faster, see http://berkshelf.com/)
The following needs to be uploaded to an S3 bucket with to be referenced by Terraform (** assumption: these files are uploaded with AWS KMS S3 master key encryption **).
- Chef validator key (for enrolling node into Chef server)
- client.rb (sample https://docs.chef.io/config_rb_client.html#example)
- File named
github_token(contains Github token string for Github role user)
Clone folder terraform of this repo and set up appropriate vars in terraform.tfvars. Specific variable explanations are below:
| Param | Explanation |
|---|---|
| aws_secret_key | Cred of power user that can create all resources Terraform needs |
| aws_access_key | Cred of power user that can create all resources Terraform needs |
| prevent_destroy | Allow Terraform to delete resources (true |
| secrets_s3_bucket | S3 bucket that contains Chef's validation key, client.rb and github_token file (encrypted with AWS KMS S3 master key) |
| ssh_key_name | Default SSH user for the base image |
| ssh_ingress | SSH CIDR range allowed to SSH into EC2 instance and access Jenkins URL |
The secrets_s3_bucket and Chef validator key are needed to be set in terraform/modules/jenkins/userdata.sh as well.
Execute following to set up:
$ cd terraform
$ terraform get
$ terraform plan
$ terraform apply
If Terraform ran successfully, the following resources would be created:
- An AutoScalingGroup for Jenkins master
- Resources associated with master instance: Security Group, IAM Role, IAM Instance Profile
- An ELB in front of the AutoScalingGroup to redirect HTTP/80 to HTTP/8080
Chef runs on instance bootstrap take several minutes on average. You can follow by ssh into the instance and tail -f /var/log/chef-client.log.
If jenkins-wrapper cookbook ran successfully, the following would be set up:
- A Jenkins master node
- Jenkins github pull request builder plugin + dependencies
- A global credential to access your Github repo via Github token
- Hashicorp Packer executable in $PATH
- A sample job to build an image using Packer
If Terraform ran successfully, you could execute terraform show | grep dns_name to see the DNS of the ELB, the access Jenkins via http://<ELB DNS name>
As of currently, Jenkins 1.6 needs these tasks to be performed manually after first time it is setup by the Chef cookbook:
- Go to 'Manage Jenkins' -> 'Configure System' and click 'Save'. This is due to error
java.io.IOException: {"message":"Validation Failed","errors":[{"resource":"Status","code":"custom","field":"target_url","message":"target_url must use http(s) scheme"}]while running Github SCM jobs the first time.
Two sample Jenkins jobs that uses Github Pull Request Builder plugin to run automated Packer builds are created by default from jenkins-wrapper cookbook (packer-build and packer-promote) Customize the job to your values (the preferrable way: setting job values using cookbook attributes and cookbook templates).
Sample Packer template and supporting scripts are available in (https://github.com/huyanhvn/image-build-pipeline/tree/master/packer). You can upload these to your repo and configure Jenkins jobs to build those. ** Important: make sure to customize the input variables of the Packerfile, especially security_group_id variable (the dev-us-east-1-demo-packer-instance-sg SG created by terraform). See explanations in README **
Issue a pull request (PR) to your Github repo setup for this pipeline.
To build the PR, one of the admins (specified by ['jenkins-wrapper']['github']['admins'] attribute in the cookbook) can comment "ok to build in dev".
To promote the built image in dev to staging/production, one of the admins can comment "promote image".
If the pipeline worked correctly, the new AMI would be built using Chef cookbook ubuntu-build, tested using serverspec, copied to other AWS regions, and shared with other AWS accounts.
Chef (https://learn.chef.io/) Jenkins cookbook (https://github.com/chef-cookbooks/jenkins) Github Pull Request Builder plugin (https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+plugin) Hashicorp Packer (https://www.packer.io/docs/) serverspec (http://serverspec.org/) Hashicorp Terraform (https://www.terraform.io/docs/)