Introduction

Welcome to the Automatics Jenkins Setup and Deployment Guide. This document will walk you through the steps to set up and deploy Automatics applications using Jenkins, ensuring a smooth and efficient process.

Automatics Automated build and deployment Repo

Repo : https://code.rdkcentral.com/r/admin/repos/rdk/tools/automatics/dev-ops

(Please open the above url in browser and git clone url is available in the page)

Branch : rdk-next

Sub folder : 

• Build : automationScripts\build\
• Deployment : automationScripts\deployment\
• Jenkins Setup : jenkinsServerSetup\

Prerequisites

Before you begin, ensure you have the following:

• Virtual Machine (VM): VMs with root access.
• Git: Installed on your Jenkins VM.
• Docker :
  • To be installed in all VMs where Automatics components has to be deployed
•  Install using the following commands:

  curl -fsSL https://get.docker.com -o get-docker.sh sh get-docker.sh

  • Create a new user group named "docker" and add the Jenkins user to this group:

    sudo groupadd docker sudo usermod -aG docker $USER

  • Log out and log back in to re-evaluate group membership. On a virtual machine, you may need to restart it. On Linux, you can also run:

    newgrp docker

  • Verify Docker commands can be run without sudo:

    docker run hello-world

  • Ensure access to pull Docker base images like Tomcat and MySQL from Docker Hub. If access is not available, ensure these images are available locally on the Jenkins VM.

Jenkins Server Setup

1. Clone the repository https://code.rdkcentral.com/r/admin/repos/rdk/tools/automatics/dev-ops(Please open the above url in browser and git clone url is available in the page) and navigate to the jenkinsServerSetup directory:

   git clone <repository_url> cd jenkinsServerSetup

   
   

2. Execute the installation script as sudo:

   sudo sh install_jenkins_server.sh

   This script installs Docker, pulls the Jenkins server image from Docker Hub, and installs necessary dependencies like Maven and Jenkins plugins. It also creates Automatics Core & RDKB Utils build jobs and build and deployment jobs.

3. During script execution, provide inputs such as the port for the Jenkins server. After setup, access Jenkins via the browser using the public IP and port (e.g., 192.168.91.44:8909).

4. Configure Jenkins security:

   • Go to "Manage Jenkins" → "Security".
   • Under Security Realm, select Jenkins’ own user database.
   • In the Authorization section, select "Logged-in users can do anything".
   • Unselect "Allow anonymous read access".
   • Click "Save".

5. Modify the necessary configurations in the pre-created Jenkins jobs (e.g., Maven repo URL, shell script file paths).

Running the jobs

Build Job

• Populate the parameters as shown in the sample images.

• Tick the checkbox like "BUILD_AUTOMATICS_ORCHESTRATION" for the orchestration build. You can tick or untick checkbox against each Automatics application to build the corresponding docker container
• Provide the repository URL, branch, and credentials for each Automatics project.
• Click "Build" to start the build.

Deployment Job

Prerequisite

1. Docker : All the VMs in which you are planning to deploy Automatics should have docker pre installed in them before executing the deployment job.

• Provide details of the VMs for deploying each tool, including user details for SSH access.
• Ensure SSH connection to the Jenkins VM for executing commands and copying Docker images.
• Select the tools to deploy and click "Build".

Mandatory Automatics Configurations for Test Execution

The below configurations needs to be available in Automatics Orchestration, Automatics Properties, Device Manager and Jenkins jobs inorder to trigger a successful Test case execution from Automatics Orchestration

Jenkins Jobs

The below set of jobs will be created after successful completion of Automated Build and Deployment execution.

Configure all the jenkins jobs with below mentioned details 

After completing the above set of configuration "CI_Automatics_Code_Deploy_And_3.0_Api_Parser" job should be executed. This job will build and deploy all the automatics dependencies to the repo and run the parsers needed for Automatics Scriptless automation (Automatics 3.0). After that "Deploy_RPi_Partner_Java_API" job needs to be executed. Once all these jobs are executed successfully it is good to go with jenkins job configuration for Automatics( Full manual configuration can be found here Jenkins Job for Test Execution#2.ConfigureJenkinsJob)

Automatics Orchestration

System Configuration

The below mandatory system configurations needs to be done in Automatics orchestration for running a first dummy Test execution and result verification from automatics. All the URLs and DB credentials used for the deployment can be found as a YAML file in "archived artifacts" of Automatics Deployment Job execution DeploymentJob

Device Details

You need to configure the below data related to device in Automatics orchestration in order to trigger a Test case successfully for the first time.

• Device Models -  Automatics Orchestration Setup#DeviceModels
• Device Groups - Automatics Orchestration User Manual#ConfigureDeviceGroup 
• Resource Details (Jenkins job settings)- Automatics Orchestration Setup#ResourceDetails
• Source Code Repository - Automatics Orchestration Setup#SourceCodeRepository

Automatics Properties

All the mandatory properties mentioned in the document Automatics Properties should be configured in automatics properties for the successful execution of a test case.

Important Points to Note

New Features in Release 33

• Docker Compose Integration: Automated deployment scripts now use Docker Compose.
• Environment Files: All Docker container data (like VM IP, port credentials, etc.) is stored in an environment file for each Automatics application stack. A stack includes both the backend server Docker container and the database Docker container.

Simplified Container Management

• Easier Start/Stop: Starting and stopping Automatics Docker containers is now more user-friendly and efficient.
• Automatic Port Assignment: You no longer need to manually identify and provide free ports in the VM. The script will automatically assign a free port to each Docker container. You can see the assigned ports in the YAML file generated under "archived artifacts" after completing the deployment job. If you input a specific port, the script will use that port for deploying the corresponding application.
• Automated Directory Creation: You no longer need to create directories in the VM and provide them as input to the Automatics deployment job. The automated scripts will use Docker volumes to create and attach volumes for each application stack, ensuring logs and database data persist even after deleting or removing Docker containers.

Upgrading from Older Versions

For users upgrading from older versions of Automatics using the latest scripts, please note the following one-time activities:

1. Stop and Remove Old Containers: Manually stop and remove old Automatics Docker containers.
2. New Containers: After running the deployment job, all containers will be new, so no existing data will be present.
3. Backup and Restore Data:
   • Backup and restore database data to the new DB Docker containers.
   • Backup and restore automatics.properties and device_config.json files to the newly deployed application.
4. Log Files: Old Automatics Docker container log files will remain in the existing VM directory. For accessing the logs you can go inside the docker container and open the logs folder, all the server logs will be available in that location. 

Upgrading Automatics Applications Post-Release

To upgrade Automatics applications after a release using the latest automated deployment scripts:

1. Identify Changes: Determine which Automatics application stacks have new changes.
2. Build Job:
   • Open the Automatics Build job in Jenkins.
   • Tick the Build button for the application stacks with changes and run the build job.
3. Deployment Job:
   • Open the Deployment Job.
   • Input all parameters, including the existing VM IP address and port where the application stack is running.
   • Run the deployment job to deploy the latest changes to the corresponding application.