1. Introduction 🌐

In the dynamic realm of today’s digital landscape, the development and deployment of applications demand not only speed but also an impenetrable layer of security. This is where DevSecOps assumes a pivotal role, seamlessly integrating development, security, and operations into a cohesive and unified process.

Within this comprehensive guide, we are set to embark on a journey where we harness the power of Terraform, Jenkins CI/CD, SonarQube, Trivy, Argocd, and Amazon Elastic Kubernetes Service (EKS) to establish a resilient and secure pipeline for deploying applications on Amazon Web Services (AWS).

Whether you are a seasoned developer seeking to augment your DevSecOps proficiency or a newcomer eager to explore the captivating intersection of software development and security, this guide holds invaluable insights for you.

Let’s delve into the intricacies and uncover the steps to fortify your Amazon application, ensuring a seamless and efficient deployment process.

2. Project Resources

GitHub Link:

github.com/mudit097/Swiggy-Clone-App — Application code.

github.com/mudit097/Swiggy-App-ArgoCD — Manifest files.

2.1. Overview 📋

• Infrastructure as Code: Use Terraform to define and manage AWS infrastructure for the application.

• Container Orchestration: Utilize Amazon EKS for managing and scaling containerized applications.

• CI/CD with Jenkins: Set up Jenkins to automate building, testing, and deploying the application.

• Static Code Analysis: Incorporate SonarQube to analyze code quality and identify vulnerabilities.

• Container Image Scanning: Integrate Trivy to scan container images for security issues.

• Application Deployment with Argocd: Use Argocd for declarative, GitOps-based application deployment on EKS.

2.2. Architecture 🏗️

3. Step-by-Step Guide

3.1. Create IAM User 👤

Navigate to the AWS console

• Create an IAM user with administration access.

• Log in to the AWS Console with the above user.

• Create one free-tier EC2 instance with Ubuntu.

• Login to the EC2 instance and follow the below steps.

3.2. AWS Configuration ☁️

Install the AWS Cli in your EC2 Ubuntu.

Install AWS CLI:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws --version

aws configure

Provide your Aws Access key and Secret Access key

3.3. Terraform Files and Provision Jenkins, Sonarqube ⚙️

Terraform Installation in an EC2 Instance:

wget https://releases.hashicorp.com/terraform/1.3.7/terraform_1.3.7_linux_amd64.zip
unzip terraform_1.3.7_linux_amd64.zip
mv terraform /usr/local/bin
sudo mv terraform /usr/local/bin
terraform -v

main.tf

resource "aws_instance" "web" {
  ami                    = "ami-0fc5d935ebf8bc3bc"   #change ami id for different region
  instance_type          = "t2.large"
  key_name               = "Mario"
  vpc_security_group_ids = [aws_security_group.Jenkins-sg.id]
  user_data              = templatefile("./install.sh", {})

  tags = {
    Name = "Jenkins-sonarqube-trivy-vm"
  }

  root_block_device {
    volume_size = 30
  }
}

resource "aws_security_group" "Jenkins-sg" {
  name        = "Jenkins-sg"
  description = "Allow TLS inbound traffic"

  ingress = [
    for port in [22, 80, 443, 8080, 9000, 3000] : {
      description      = "inbound rules"
      from_port        = port
      to_port          = port
      protocol         = "tcp"
      cidr_blocks      = ["0.0.0.0/0"]
      ipv6_cidr_blocks = []
      prefix_list_ids  = []
      security_groups  = []
      self             = false
    }
  ]

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "jenkins-sg"
  }
}

provider.tf

#provider.tf

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "us-east-1"  #change your region
}

install.sh

This will install Jenkins, Docker, Sonarqube, and Trivy by Terraform with an EC2 instance.

#!/bin/bash
sudo apt update -y
wget -O - https://packages.adoptium.net/artifactory/api/gpg/key/public | tee /etc/apt/keyrings/adoptium.asc
echo "deb [signed-by=/etc/apt/keyrings/adoptium.asc] https://packages.adoptium.net/artifactory/deb $(awk -F= '/^VERSION_CODENAME/{print$2}' /etc/os-release) main" | tee /etc/apt/sources.list.d/adoptium.list
sudo apt update -y
sudo apt install temurin-17-jdk -y
/usr/bin/java --version
curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null
echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y
sudo apt-get install jenkins -y
sudo systemctl start jenkins
sudo systemctl status jenkins

#install docker
sudo apt-get update
sudo apt-get install docker.io -y
sudo usermod -aG docker ubuntu  
newgrp docker
sudo chmod 777 /var/run/docker.sock
docker run -d --name sonar -p 9000:9000 sonarqube:lts-community

#install trivy
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y

Terraform commands to provision:

terraform init

terraform validate

terraform plan

terraform apply

The EC2 instance Jenkins-sonarqube-trivy-vm is created by Terraform with Jenkins, Sonarqube, and Trivy as userdata for the EC2 instance, which is installed during the creation of the EC2 instance.

Output:

Take the public IP address of the EC2 instance, as shown in the below image.

<Public IPV4 address>:8080. #For accessing Jenkins

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Unlock Jenkins using an administrative password and install the suggested plugins.

Jenkins will now get installed and install all the libraries.

Create a user, click save, and continue.

Jenkins Getting Started Screen.

Copy your public key again and paste it into a new tab.

<instance-public-ip>:9000

Enter your username and password, click on login, and change your password.

username admin
password admin

Update the new password. This is the Sonar Dashboard, as shown below.

Check Trivy version

Check the Trivy version in an Ec2 instance.

trivy --version

4. Install Plugins like JDK, Sonarqube Scanner, NodeJs, and OWASP Dependency Check

4.1. Install Plugin 🛠️

Goto Manage Jenkins →Plugins → Available Plugins

Install below plugins

1. Eclipse Temurin Installer (Install without restart)
2. SonarQube Scanner (Install without restart)
3. NodeJs Plugin (Install without restart)
4. Sonar Quality Gates (Install without restart)

4.2. Configure Java and Nodejs in Global Tool Configuration 🛠️

Goto Manage Jenkins → Tools → Install JDK (17) and NodeJs (16). Click on Apply and Save

Choose the option install from adoptium.net

5. Configure Sonar Server in Manage Jenkins ⚙️

Grab the public IP address of your EC2 instance.

Sonarqube works on Port 9000, so <Public IP>:9000.

Go to your Sonarqube server.

Click on Administration → Security → Users → Click on Tokens and Update Token, → Give it a name, and click on Generate Token

click on update Token

Create a token with a name and generate

copy Token

Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add secret text. It should look like this

You will see this page once you click on create

Now, go to Dashboard → Manage Jenkins → System and add like the below image.

Click on Apply and Save.

The Configure System option is used in Jenkins to configure different server

Global Tool Configuration is used to configure different tools that we install using Plugins

We will install a sonar scanner in the tools.

In the Sonarqube Dashboard, add a quality gate as well.

In the sonar interface, create the quality gate as shown below:

Click on the quality gate, then create.

Click on the save option.

In the Sonarqube Dashboard, Create Webhook option as shown in below:

Administration → Configuration →Webhooks

Click on Create

Add details:

<http://jenkins-private-ip:8080>/sonarqube-webhook/

Let’s go to our pipeline and add the script to our pipeline script.

pipeline {
    agent any
    tools {
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace') {
            steps {
                cleanWs()
            }
        }
        stage('Checkout from Git') {
            steps {
                git branch: 'main', url: 'https://github.com/mudit097/Swiggy-Clone-App.git'
            }
        }
        stage("Sonarqube Analysis ") {
            steps {
                withSonarQubeEnv('sonar-server') {
                    sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Swiggy-CICD \
                    -Dsonar.projectKey=Swiggy-CICD '''
                }
            }
        }
        stage("quality gate") {
            steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token' 
                }
            } 
        }
        stage('Install Dependencies') {
            steps {
                sh "npm install"
            }
        }
    }
}

Click on Build now, and you will see the stage view like this:

To see the report, you can go to Sonarqube Server and go to Projects.

You can see the report has been generated, and the status shows as passed. You can see that there are 765 lines it has scanned. To see a detailed report, you can go to issues.

6. Install OWASP Dependency Check Plugins 🧐

Go to Dashboard → Manage Jenkins → Plugins → OWASP Dependency-Check. Click on it and install it without restarting.

First, we configured the plugin, and next, we had to configure the Tool

Goto Dashboard → Manage Jenkins → Tools →

Click on Apply and save here.

Now go to Configure → Pipeline and add this stage to your pipeline and build.

stage('OWASP FS SCAN') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('TRIVY FS SCAN') {
            steps {
                sh "trivy fs . > trivyfs.txt"
            }
        }

You will see that in status, a graph will also be generated for vulnerabilities.

7. Docker Image Build and Push 🐳

We need to install the Docker tool on our system.

Go to Dashboard → Manage Plugins → Available plugins → Search for Docker and install these plugins.

Now, goto Dashboard → Manage Jenkins → Tools →

Now go to the Dockerhub repository to generate a token and integrate with Jenkins to push the image to the specific repository.

If you observe, there is no repository related to Swiggy.

There is an icon with the first letter of your name.

Click on that My Account, → Settings → Create a new token and copy the token.

Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add secret text. It should look like this:

Add this stage to Pipeline Script.

stage("Docker Build & Push"){
            steps{
                script{
                   withDockerRegistry(credentialsId: 'dockerhub', toolName: 'docker'){ 
                    app.push("${env.BUILD_NUMBER}")  
                       sh "docker build -t swiggy-app ."
                       sh "docker tag swiggy-app mudit097/swiggy-app:latest "
                       sh "docker push mudit097/swiggy-app:latest "
                    }
                }
            }
        }
        stage("TRIVY"){
            steps{
                sh "trivy image mudit097/swiggy-app:latest > trivyimage.txt" 
            }
        }

You will be able to view the output in the Jenkins pipeline and output upon successful execution.

When you log in to Dockerhub, you will see a new image is created.

8. Creation of EKS Cluster with ArgoCD 🌐

EKSCTL Installation:

Now let’s install EKSCTL in Ubuntu EC2 which was created earlier.

ARCH=amd64
PLATFORM=$(uname -s)_$ARCH
curl -sLO "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
sudo mv /tmp/eksctl /usr/local/bin

# Check the eksctl version.
eksctl version

Command to Create EKS Cluster using eksctl command:

eksctl create cluster --name <name-of-cluster> --nodegroup-name <nodegrpname> --node-type <instance-type> --nodes <no-of-nodes>

eksctl create cluster --name my-eks-cluster --nodegroup-name ng-test --node-type t3.medium --nodes 2

It will take 5–10 minutes to create a cluster.

As you will see in the EC2 instances running list one instance is running in the name of EKS Cluster as shown below.

EKS Cluster is up and ready and check with the below command.

Now let’s install ArgoCD in the EKS Cluster.

kubectl create ns Argocd
# This will create a new namespace, argocd, where Argo CD services and application resources will live.
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Download Argo CD CLI:

curl -sSL -o argocd-linux-amd64 https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
sudo install -m 555 argocd-linux-amd64 /usr/local/bin/argocd

Access The Argo CD API Server:

# By default, the Argo CD API server is not exposed with an external IP. To access the API server, 
choose one of the following techniques to expose the Argo CD API server:
* Service Type Load Balancer
* Port Forwarding

Let’s go with Service Type Load Balancer.

# Change the argocd-server service type to LoadBalancer.
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

List the resources in the namespace:

kubectl get all -n argocd

Get the load balancer URL:

kubectl get svc -n argocd

Pickup the URL and paste it into the web to get the UI as shown below image:

Login Using The CLI:

argocd admin initial-password -n argocd

Login with the admin and Password in the above you will get an interface as shown below:

Click on New App:

Enter the Repository URL, set path to ./, Cluster URL to kubernetes.default.svc, the namespace to default and click save.

The GitHub URL is the Kubernetes Manifest files which I have stored and the pushed image is used in the Kubernetes deployment files.

Repo Link: https://github.com/mudit097/Swiggy-App-ArgoCD

You should see the below, once you’re done with the details.

Click on it.

You can see the pods running in the EKS Cluster.

We can see the out-of-pods using the load balancer URL:

kubectl get svc

With the above load balancer, you will be able to see the output as shown in the below image:

As you observe in the above image, I just want to change the address of the Swiggy Application.

Then the real magic will happen, the changes updated in the code we will try to push the code to GitHub and run a pipeline to push the image to the repository with the updated details.

I will click on the sync option which is in the ArgoCD and then the updates will be in our Swiggy Website.

I have changed the website details of the address in the code and then you can see the results in the below image.

Conclusion:

In this project, we have covered the essential steps to deploy a Swiggy app with a strong focus on security through a DevSecOps approach. By leveraging tools like Terraform, Jenkins CI/CD, SonarQube, Trivy, Argocd, and EKS, we can create a robust and secure pipeline for deploying applications on AWS.

Remember that security is an ongoing process, and it is crucial to stay updated with the latest security practices and continuously monitor and improve the security of your applications. With the knowledge gained from this guide, you can enhance your DevSecOps skills and ensure the smooth and efficient deployment of secure applications on Amazon Web Services.

Thank you for reading this post! I hope you found it helpful. If you have any feedback or questions, Please connect with me on LinkedIn at

https://www.linkedin.com/in/mudit--mathur/

mudit097 - Overview

Linux || Docker || AWS || GCP || Azure || Jenkins || K8S || Ansible || Terraform || - mudit097

github.com

Your feedback is valuable to me. Thank you!