2023 Checklist for Production Deployment on AWS

[Last updated on 07/26/2023]

#Continuous Integration

Continuous integration is the process of frequently integrating the code so that any code conflicts and code breakage can be found and fixed immediately. Here are some critical points which must be ensured before going live:

• Perform proper code reviews and implement a proper PR approval process. That includes a review of every pull request as well as scheduled code reviews by peers or by senior members.
• Adopt a trunk-based git workflow like Github flow instead of legacy git-flow
• Properly define all the dependencies of your application, along with the exact version of each dependency. You can use docker to handle all the dependencies if your application is containerized
• Use linters and code coverage reports on the code being submitted by developers.
• Setup automated unit tests as part of the development. It may increase your overall development timeline, but it will result in a much more stable application. 
• Publish your project binaries such that the application and its dependencies are formed into a deployable artifact or a container. The final artifact or container should be assigned a unique version ID to identify the artifact, e.g., Tag docker image with sprint number. Make sure to push the final artifact in the artifacts repository, e.g., Push Docker image to AWS ECR repository  
• Test environments (UAT, for example) should mirror production as much as possible. Whether you perform manual testing on final builds or automated testing, keeping the non-production environment as close to production will make your testing as real as possible.
  You can check out the clone environment feature of Qovery, which can be very helpful in this regard.
• Make sure that neither the deployment artifact (i.e., container) nor the environment configuration (.env files) is part of the source code repository. These are considered metadata and not part of the code. 

#Continuous Delivery

Where continuous integration focuses on continuous integration of code, the process of continuous delivery ensures the builds are deployed as soon as the code is pushed/merged. The testing team can review the latest application upon every commit. Here are some of the points which every modern application should adopt:

• Deployment environments should be separate from each other. In the case of AWS, prefer to have each environment a separate VPC (for small to medium teams) or in a separate account (for large teams or compliance sensitive projects). If you use Kubernetes for container orchestration, you should check our article regarding how to isolate production cluster from staging cluster. 
• While the code and dependencies are centralized, the configuration for each environment should vary from environment to environment. This is the same point that was highlighted in the “Configuration” area of our blog article related to twelve-factor here
• Do not define your deployment infrastructure manually e.g., through AWS console or AWS CLI. Try to set up Infrastructure as code (IaC). A few examples are Terraform and AWS CloudFormation. You should also create automated tests to verify the setup created through IaC. 
• Deploy the same container or deployment artifact sequentially to all the environments. E.g., if you deploy version 0.1 to development and if it passes the tests, then same version 0.1 should be promoted to staging, and then the same version 0.1 should be promoted to production if it passes staging. It is the same code and set of dependencies but with different configurations in each environment. 
• Automate all your deployments. You can achieve it through pipelines, and AWS provides tools like AWS code build and Code deploy etc. You can also use Github actions, Gitlab CI or some automation tools like CircleCI 
• Aim for zero downtime in your deployments. You can use many techniques to achieve zero downtime deployment—some commonly used techniques include rolling updates, blue-green deployment etc.. 
• Ensure that an automated test suite is in place for automated deployment. These are not developer-written unit tests. These are test cases written by SQA engineers through automation testing tools like selenium web driver. You should also set up an email notification if automates tests were passed or failed on the deployed artifact. 

#Security

Secure DevOps is a crucial part of every DevOps these days. Security must be added to all the stages of the SDLC and not just to the deployment. Find below some of the checklist points related to security in the whole process of CI/CD:

• Secure all network connections. If you are on AWS, you can use ACM to create SSL certificates, and it is very easy to apply these certificates to AWS components like Elastic Load Balancer, CloudFront etc. 
• Apart from securing network connections, make sure the encryption at rest is also enabled. AWS provides a built-in encryption feature at rest for most of its popular services, including EC2, S3 bucket, etc. 
• Assuming your application is containerized, make sure you are following all the guidelines mentioned in our article related to container security 
• Make sure all API keys, secrets, etc. are stored securely. No secrets should be stored in plain text. You can use some built-in service provided by the cloud e.g., AWS SSM secrets manager, etc. 
• Implement Security as code (SaC) and secure coding practices. Perform regular code scanning for new security recommendations. Keep OWASP top ten security vulnerabilities in mind for SaC. 
• Make sure all your cloud VMs are on a private subnet (especially databases), and you can access them only through a bastion host. Implement proper whitelisting of IPs for incoming connections. 
• Make sure you implement both Static Application Security Testing (SAST) and Dynamic Application Scanning (DAST) to ensure security of both your source code and the application running in its environment. 
• Implement proper role-based access i.e., by using AWS IAM roles to restrict access to deployment servers, databases, source code repository, build servers, etc. Aim for the least privileged IAM policies if using AWS. 
• Make sure that we apply hardening to the host and containers too. Perform regular security assessment of your VMs, containers and any infrastructure in place (both on cloud and on-premises)

#Monitoring

Monitoring is crucial for your production deployments. Not just to make sure the deployments are successful but also to troubleshoot and fix any possible problems. Here are some essential points which are must-haves in this checklist:

• Track application metrics, especially the application load, error logs, throughput, latency, etc. You can use tools like Amazon CloudWatch and Datadog etc.  
• Keep an eye on your infrastructure's health, including your VMs, databases, etc. Some critical metrics include available memory, available disk space, CPU, etc. You can use the same tools mentioned in the previous point.
• Keep an eye on the logs. That includes not just the error logs but the debug logs too. Sometimes the logs indicate a meaningful pattern of events that can be used to catch and fix a problem proactively. Some notable tools include AWS CloudWatch, Papertrail, etc.
• Setup notification alerts. E.g., If the CPU of your VM goes beyond 80%, then send an email to important stakeholders. AWS CloudWatch alarms can be set up and integrated with AWS SNS for email deliveries of important notifications related to the health of your application and infrastructure. 

#Conclusion

In this article, we have gone through a checklist that will ensure your product deployment is production-ready on AWS. We discussed various checklist points related to continuous integration, continuous deployment, monitoring, and security. As you would have noticed that it is not easy to keep pace with the dynamic and complex needs of CI/CD. This is where Qovery can help you. With Qovery, even a novice in the tech industry can deploy apps on AWS in mere minutes. Using your own AWS account, you can use Qovery to scale your team and manage your cloud infrastructure, deployments, optimize cloud cost, etc., with great simplicity. Try Qovery for free!