Introduction

Welcome to the Automatics Jenkins Setup and Deployment Guide. This document will walk you through the steps to set up and deploy Automatics applications using Jenkins, ensuring a smooth and efficient process.

Deployment Approaches

Deployment Setup in 3 VM

The recommended approach to deploy Automatics for a stable test environment is using 3 VM setup. In this approach the automatics components are deployed among the VMs. Following diagram shows in which VM each component should be deployed.

Deployment Setup in single VM 

In this approach all components that is shown in the above section will be deployed in single VM.

Automatics Setup Steps

• Pre-requisites

Before you begin, ensure you have the following installed in VMs:

• Virtual Machine (VM): VMs with root access.

• Git: Installed on your Jenkins VM.

• Docker : To be installed in all VMs where Automatics components has to be deployed

• Self Hosted Maven Artifact Repository : (Optional)

  • This is where Automatics maven artifacts will be deployed and used during test execution using maven. If this setup is not available, please press enter for inputs asking maven artifact repo details. If planning to keep the artifacts in maven local repo in the same machine, please replace the command clean deploy with clean install in jenkins jobs configuration where ever applicable.

Docker Setup Steps

Please follow below steps to setup docker in your VMs.

curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

Create a new user group named "docker" and add the Jenkins user to this group:

sudo groupadd docker
sudo usermod -aG docker $USER

Log out and log back in to re-evaluate group membership. On a virtual machine, you may need to restart it. On Linux, you can also run:

newgrp docker

Verify Docker commands can be run without sudo:

docker run hello-world

Note: Ensure access to pull Docker base images like Tomcat and MySQL from Docker Hub. If access is not available, ensure these images are available locally on the Jenkins VM.

• Clone Automated Deployment Repository

After the pre-requisites are done, clone the automated deployment repository in the VM where Jenkins is planned to setup.

Automatics Automated build and deployment Repo

Repo : https://code.rdkcentral.com/r/admin/repos/rdk/tools/automatics/dev-ops

(Please open the above url in browser and git clone url is available in the page)

Branch : rdk-next

Sub folder : 

• • • • Build : automationScripts\build\
      • Deployment : automationScripts\deployment\
      • Jenkins Setup : jenkinsServerSetup\

Clone the repository https://code.rdkcentral.com/r/admin/repos/rdk/tools/automatics/dev-ops(Please open the above url in browser and git clone url is available in the page) and navigate to the jenkinsServerSetup directory:

git clone <repository_url>
cd dev-ops/jenkinsServerSetup

• Setup Automatics Jenkins

1. Execute the installation script as sudo from dev-ops/jenkinsServerSetup:

   sudo sh install_jenkins_server.sh

   This script installs Docker, pulls the Jenkins server image from Docker Hub, and installs necessary dependencies like Maven and Jenkins plugins. It also creates Automatics Core & RDKB Utils build jobs and build and deployment jobs.

2. During script execution, provide inputs such as the port for the Jenkins server. After setup, access Jenkins via the browser using the public IP and port (e.g., 192.168.91.44:8909).

3. Configure Jenkins security:

   • Go to "Manage Jenkins" → "Security".
   • Under Security Realm, select Jenkins’ own user database.
   • In the Authorization section, select "Logged-in users can do anything".
   • Unselect "Allow anonymous read access".
   • Click "Save".

4. Modify the necessary configurations in the pre-created Jenkins jobs (e.g., Maven repo URL, shell script file paths).

5. Add the user credential under Manage Jenkins->Credentials->System→Global Credentials.
   

• Execute Jenkins Jobs to Deploy Automatics

First execute Automatics_Build_Job. This will build the source code of Automatics tools and services and generate war or jar files. 

Populate the parameters as shown in the sample images.

• Tick the checkbox like "BUILD_AUTOMATICS_ORCHESTRATION" for the orchestration build. You can tick or untick checkbox against each Automatics application to build the corresponding docker container
• Provide the repository URL, branch, and credentials for each Automatics project.
• Click "Build" to start the build.

Next, execute Automatics_Deployment_Job. This will create docker files and deploy dockers on the destination VMs. After this execution, Automatics tools should be up and running.

While running this job, 

• Provide details of the VMs for deploying each tool, including user details for SSH access.
• Ensure SSH connection to the Jenkins VM for executing commands and copying Docker images.
• Select the tools to deploy and click "Build".

The below set of jobs will be created after successful completion of Automated Build and Deployment execution.

• Jenkins Job Configuration

Configure all the jenkins jobs with below mentioned details 

After completing the above set of configuration execute Jenkins Jobs in below order.

1.  "CI_Automatics_Code_Deploy_And_3.0_Api_Parser".  
   1. This job will build and deploy all the automatics dependencies to the repo and run the parsers needed for Automatics Scriptless automation (Automatics 3.0). 
2.  "Deploy_RPi_Partner_Java_API"
   1.  This job will build and deploy rpi provider partner implementation.

Automatics Orchestration

System Configuration

The below mandatory system configurations needs to be done in Automatics orchestration for running a first dummy Test execution and result verification from automatics. Please follow Automatics Orchestration Setup#Configuration to configure required configs System Config page of Automatics Properties to make use of features like scriptless automation, review management etc

 All the URLs and DB credentials used for the deployment can be found as a YAML file in "archived artifacts" of Automatics Deployment Job execution DeploymentJob

Device Details

You need to configure the below data related to device in Automatics orchestration in order to trigger a Test case successfully for the first time.

• Device Models -  Automatics Orchestration Setup#DeviceModels
• Device Groups - Automatics Orchestration User Manual#ConfigureDeviceGroup 
• Resource Details (Jenkins job settings)- Automatics Orchestration Setup#ResourceDetails
• Source Code Repository - Automatics Orchestration Setup#SourceCodeRepository

Automatics Properties

All the mandatory properties mentioned in the document Automatics Properties should be configured in automatics properties for the successful execution of a test case.

After all configuration are done, please login to the VM where debugger service is running and restart the debugger docker container.

Important Points to Note

New Features in Release 33

• Docker Compose Integration: Automated deployment scripts now use Docker Compose.
• Environment Files: All Docker container data (like VM IP, port credentials, etc.) is stored in an environment file for each Automatics application stack. A stack includes both the backend server Docker container and the database Docker container.

Simplified Container Management

• Easier Start/Stop: Starting and stopping Automatics Docker containers is now more user-friendly and efficient.
• Automatic Port Assignment: You no longer need to manually identify and provide free ports in the VM. The script will automatically assign a free port to each Docker container. You can see the assigned ports in the YAML file generated under "archived artifacts" after completing the deployment job. If you input a specific port, the script will use that port for deploying the corresponding application.
• Automated Directory Creation: You no longer need to create directories in the VM and provide them as input to the Automatics deployment job. The automated scripts will use Docker volumes to create and attach volumes for each application stack, ensuring logs and database data persist even after deleting or removing Docker containers.

Upgrading from Older Versions

For users upgrading from older versions of Automatics using the latest scripts, please note the following one-time activities:

1. Stop and Remove Old Containers: Manually stop and remove old Automatics Docker containers.
2. New Containers: After running the deployment job, all containers will be new, so no existing data will be present.
3. Backup and Restore Data:
   • Backup and restore database data to the new DB Docker containers.
   • Backup and restore automatics.properties and device_config.json files to the newly deployed application.
4. Log Files: Old Automatics Docker container log files will remain in the existing VM directory. For accessing the logs you can go inside the docker container and open the logs folder, all the server logs will be available in that location. 

Upgrading Automatics Applications Post-Release

To upgrade Automatics applications after a release using the latest automated deployment scripts:

1. Identify Changes: Determine which Automatics application stacks have new changes.
2. Build Job:
   • Open the Automatics Build job in Jenkins.
   • Tick the Build button for the application stacks with changes and run the build job.
3. Deployment Job:
   • Open the Deployment Job.
   • Input all parameters, including the existing VM IP address and port where the application stack is running.
   • Run the deployment job to deploy the latest changes to the corresponding application.