Introduction

Edstem undertook the task of migrating some complex existing Jenkins CI/CD pipelines of a client to Github actions. This guide aims to share the know-how obtained during the process.

Migrating Continuous Integration and Continuous Deployment (CI/CD) pipelines from Jenkins to GitHub Actions offers several advantages, including cost savings and a more flexible and secure hybrid model. Jenkins deployments typically uses servers hosted in their own data centers, while Github actions offer a hybrid model where self-hosted runners from DevSecOps can be used for secure job workflows and the open runners for other common workflows, which handled less sensitive information/code.

This guide provides a step-by-step approach to migrate your pipelines to GitHub Actions while addressing specific scenarios, such as migrating from SVN to Git and integrating build servers. The guide follows some specific configs that Edstem tackled to achieve the feat, but it can be adapted to your own custom setups. By following this guide, you'll be able to seamlessly transition your workflows and take advantage of the powerful features offered by GitHub Actions.

Table of Contents

1. Common Recommendations
   1. SVN to GitHub Migration
2. Integration Build Server Mode 1
   1. Existing Jenkins Setup
   2. Migration Steps
   3. GitHub Workflow Steps
3. Integration Build Server Mode 2
   1. Existing Jenkins Setup
   2. Migration Steps
   3. GitHub Workflow Steps
4. Able Server
   1. Existing Jenkins Setup
   2. Migration Steps
   3. GitHub Workflow Steps
5. Github Actions : Limitation
   1. User input prompt during the workflow run
   2. Dynamic input for running workflow
6. Conclusion

1. Common Recommendations

1.1 SVN to GitHub Migration

In our case Jenkins jobs was using SVN as version control system, instead of git. If you existing repositories use SVN, to ensure seamless integration with GitHub Actions, it is highly recommended to migrate your repositories from SVN to Git.

NOTE: If the jobs you're planning to migrate already uses git, you can ignore this section.

Rather than performing a manual migration, the recommended approach involves developing a custom command-line application that automates the migration process. This application should perform the following steps:

• Prompt for existing SVN repository and new GitHub repository details
• Checkout the SVN repository to a temporary location
• Create a new Git repository and push the code to GitHub using the GitHub APIs

For more detailed guidance on how to migrate from SVN to Git, refer to the provided reference: How to Migrate from SVN to Git

2. Integration Build Server Mode 1

2.1 Existing Jenkins Setup

In this mode, Jenkins jobs are executed by running Apache Ant scripts using the TIBant plugin. TIBant provides a set of Apache Ant tasks for building and deploying TIBCO BusinessWorks projects.

Example Job

2.2 Migration Steps

To migrate this setup to GitHub Actions, follow these steps:

DevOps Setup

• Create a self-hosted secure runner for GitHub Actions, preferably using Ubuntu OS. Hosting your own runner
• Configure the environment variables used in the common.xml properties tag.
• Install Apache Ant version 1.8.1 to 1.9.3 (TIBant supported version) on the runner.
• Place the attached ANT common script XML file in the runner and update the location in the build.xml file in the repository.
• Ensure connectivity from the runner to SVN if you are still using SVN as the version controller.
• Install and configure the required tools and software (TIBCO, IBM WebSphere, Soap UI) on the runner, updating the location in build.xml and common.xml.

GitHub Workflow Steps

Configure the following workflow steps in your GitHub Actions configuration file:

on:
  workflow_dispatch:
  
jobs:
  build-and-publish:
    runs-on: [self-hosted]

    steps:
     - uses: actions/checkout@v2
     - name: build
       run: ant build
     - name: publish
       run: ant publish

3. Integration Build Server Mode 2

3.1 Existing Jenkins Setup

In this mode, Jenkins jobs are used to create infrastructure using bash scripts with AWS commands.

Example Job

3.2 Migration Steps

To migrate this setup to GitHub Actions, follow these steps:

DevOps Setup

• Create a self-hosted secure runner for GitHub Actions, preferably using Ubuntu OS. Hosting your own runner
• Check if the existing non-prod/prod self-hosted runners can be reused instead of creating new runners.
• Install the AWS CLI on the runner.
• Configure the AWS CLI with the service account having the required permissions.
• Configure the environment variables used in the existing Jenkins server.

GitHub Workflow Steps

Configure the following workflow steps in your GitHub Actions configuration file:

on:
  workflow_dispatch:
  
jobs:
  run-shell-script:
    runs-on: [self-hosted]

    steps:
      - name: run script
        run: |
          aws cloudformation create-stack --stack-name \
          ${STACK_NAME} --template-body cloudformation/wso2esb/sns_topic.yaml \
          --parameters variables.json 

4. Able Server

4.1 Existing Jenkins Setup

In this mode, all Jenkins jobs run by executing shell scripts residing in the GitHub repository.

4.2 Migration Steps

To migrate this setup to GitHub Actions, follow these steps:

DevOps Setup

• Create a self-hosted runner for GitHub Actions, preferably using Ubuntu OS.
• Install the required tools and software (Oracle, SQLPlus, Apache, wsadmin) on the runner.
• Configure the environment variables used in the existing Jenkins server.

GitHub Workflow Steps

Configure the following workflow steps in your GitHub Actions configuration file:

on:
  workflow_dispatch:
  
jobs:
  run-shell-script:
    runs-on: [self-hosted]

    steps:
      - name: run script
        run: |
          chmod +x ./public/scripts/test.sh 
          ./deploy/scripts/test.sh

For scheduled runs, use the following example workflow:

on:
  schedule:
    - cron: '30 5,17 * * *'
  
jobs:
  run-shell-script:
    runs-on: [self-hosted]

    steps:
      - name: run script
        run: |
          chmod +x ./public/scripts/test.sh 
          ./deploy/scripts/test.sh

NOTE: If the Able application is running on the Jenkins server box, you can upload the artifact from the GitHub runner to the application server using the scp put command, followed by an ssh command to restart the server. Establish a secure connectivity between the runner and the application server using an SSH public/private key pair.

5. GitHub Actions limitations

While migrating from Jenkins to Github actions brings many benefits, there are also some limitations.

5.1 User input prompt during the workflow run

Currently, GitHub actions are not able to collect the user input values during a workflow run. So, we need to use workflow_dispatch inputs to collect user data at the beginning of a run.

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Define env name'     
        required: true
        default: 'prod'
      branch:
        description: 'Define branch name'     
        required: true
        default: 'master
jobs:
  printInputs:
    runs-on: ubuntu-latest
    steps:
    - run: |
        echo "Env: ${{ github.event.inputs.environment }}" 
        echo "Branch: ${{ github.event.inputs.branch }}"

5.2 Dynamic input for running workflow

As of now GitHub only supports static input from the user to trigger workflow. But some of the pipelines required dynamic input like EC2 instance names, AMI names etc.

See the discussion related to this here - workflow_dispach to have dynamic inputs parameters · Discussion #11795

There is one suggestion to achieve this functionality by creating two workflows, one is for getting the dynamic values and other is the real job running based on the values fetched.

Reference: Dynamic Manual Workflow · GitHub

6. Conclusion

By following the detailed steps provided in this guide, you can successfully migrate your Jenkins pipelines to GitHub Actions. Leveraging GitHub Actions' features and flexibility, you can optimize your CI/CD workflows and reduce infrastructure management costs. Remember to customize the steps according to your specific requirements and consult the provided references for further guidance. Happy migrating!