Git-Jenkins CI using Vagrant and Docker

GitHub login/signup

GitHub

Did you ever feel bored doing the same repetitive tasks over and over again such as testing and deploying code changes? Many of us developers and sysadmins experience this same scenario and if you don’t know a way to get around this, your productivity is probably very low. Fortunately, computers doesn’t feel bored doing the same things over and over. So, let’s make them do the boring stuff while we focus on the interesting ones.

In this project, we will be using Vagrant (with VirtualBox as provider) and Docker as our testing and deployment platform. We will also be using Jenkins as our CI/CD application together with GitHub. To make our setup complete, we will also create a simple NodeJS application which will serve the application to be deployed and tested.

To get started, we’ll need to install VirtualBox and Vagrant into our host first. You can head over to virtualbox.org and vagrantup.com for the installation of VirtualBox and Vagrant respectively. And don’t forget to install a git client of course! Once you install those two successfully, it’s now time to setup our repository. Head over to GitHub and sign in or sign up if you have no account yet.

After signing in/signing up, create a new repository and fill up the fields.

GitHub new repo

GitHub new repo

Then, clone your repo into your machine by opening a terminal and running git clone https://github.com/username/repo-name.git on a directory where you want your project to be placed. It should clone fairly fast since your repo contains nothing yet. Let’s fill that in by changing directory into your repo and running vagrant init ubuntu/trusty64. This should create a Vagrantfile with default values and set Ubuntu Trusty x64 as the base box. The Vagrantfile is the configuration file that Vagrant will be using to provision our server as well as the Jenkins docker container. After running the command, open the Vagrantfile using your text editor and add/edit the following:

  • Forwarded port mapping
    On the forwarded port mapping section, add the following:

Port 8080 will be used by our Jenkins installation and port 8000 for our sample app.

  • VM customization
    Add/edit the following on the config.vm.provider section:


You can adjust the values according to the capacity of your host machine.

  • Provisioner
    Next to setup is our Jenkins application. Since we will be using Docker to run the app, we might as well use Vagrant’s Docker provisioner to automate things up. Please be informed though that Vagrant’s Docker provisioner takes quite a while to install Docker into the system. It has been reported on Vagrant’s GitHub repo but no solution yet at the time of writing. We will be using Jenkin’s official Docker image which can be found at https://hub.docker.com/_/jenkins/. Before the outermost end code block, add the following:


The first line tells the provisioner to use docker. The second line tells the Docker provisioner to use run a container using the jenkinsci/jenkins image. The third line contains basically a normal docker run arguments. If you notice, we are using the --network="host" option. This is because we will be ssh’ing to our VM from the jenkins Docker container in order to deploy our app into another Docker container. One way to expose our VMs ports to our container is to use the --network="host" option.

At this point, we are ready to test if our jenkins container will run without issues. Save the Vagrantfile you just edited and close it. On your terminal, run “vagrant up” making sure your CWD is where your Vagrantfile is. You might want to grab a cup of coffee or read more about Docker here. while waiting for the build to finish. Once your VM has booted and finished provisioning the Jenkins container, ssh into your VM by running vagrant ssh (make sure you have an ssh client installed) and then run docker logs jenkins in order to obtain the password that Jenkins provided to continue the installation on the browser. The password should look like the one below:

Jenkins Password

Jenkins Password

Copy the password, open your browser and browse http://localhost:8080. You should now be looking at your Jenkins Getting started page which will ask for a password. Paste the password that you just copied and click Continue.

Unlock Jenkins

Unlock Jenkins

On the next page that show, click the Install Suggested Plugins. This will install all the necessary plugins we will be needing in setting up the deployment process.

Jenkins Suggested Plugins

Jenkins Suggested Plugins

Once the plugin installation is complete, you will be presented with a page asking for the first admin user details. Fill up the fields and click Save and Finish.

Jenkins First Admin User

Jenkins First Admin User

Now we are done with installing Jenkins. But we still don’t have an app that we can deploy and test. So let’s go and create one. On the same directory containing your Vagrantfile, create a directory named app. Inside the app directory, create a file named package.json and enter the following:

This file contains information about our NodeJS app including its dependencies. We will be using express as our application framework. Mocha and supertest will be used to test our application. After saving package.json, it’s time to create our actual application. Create an app.js file and add the following:

Save and close the file. As you can see, our app is very simple. When someone accesses the root URL of your app, it will print a simple “Hello World!” in your browser. Now that we have our app ready, let’s go ahead and Dockerize it by creating a file named Dockerfile. Inside your Dockerfile, enter the following:

Save and close the file.

The Dockerfile is the configuration file that Docker uses when building a Docker image. Now, before letting Jenkins do the actual job of running the container, we have to make sure that it will run successfully first by manually building it and running it inside our VM. To do this, on your terminal, change directory to where your Vagrantfile is located. Then run vagrant ssh. This command will create an ssh connection into our VM so we can run commands directly inside it such as docker build and run commands. Once you successfully connected to your VM, chage directory to /vagrant/app. This directory should contain a few files we created earlier such as the package.json, app.js and out Dockerfile. Let’s go ahead and build our app’s Docker image by running docker build -t app .. You don’t want to forget the dot (.) at the end of the command. This command will print out a bunch of texts like Step 1, npm info, etc. Once the command finishes, you should now have a Docker image of your app available in your local repository. You can check all the Docker images you have locally by running docker images. You will also see a node:4-onbuild image on the list since Docker downloaded this image and used it as a base for our app. You will see how it’s used by looking at the first line of the Dockerfile.

Now that we have an image of our app, let’s create a container and run it by running docker run -d -p 8000:8000 --name app-container app on your terminal. Fire up your browser and browse http://localhost:8000. You should see a “Hello World!” printed on the page which means our app is running well using Docker. Before we proceed with the actual deployment using Jenkins, let’s create some tests against our app and run it by recreating our app’s Docker image and re-running the container. On the directory where your Dockerfile is, create a directory named test. Inside it, create two files named test.js and test.sh. On the test.js file, add the following:

This is just a simple test to check if our app is indeed showing the “Hello World!” string. Save the file and open test.sh and add the following:

This simple script uses mocha to run our test.js script. Save the file and close it. Let’s rebuild our image but before that, edit your Dockerfile first and add the following right after the RUN npm install line:

Save and close the file and run docker build -t app . again to build the image. You will notice that at the botton of the build you will see something like:

That means our test is working and it passed. If you try editing the app.js and change ‘Hello World!’ to ‘Hello Earth!’ you will get something like:

Which clearly means our test failed.

Assuming we have everything correct and our build is successful let’s now re-run our container by running the same command we issued above docker run -d -p 8000:8000 --name app-container app. I’m pretty sure you got an error there. That’s because in Docker, you cannot run containers with the same name and if the previous container is still running, the ports will also conflict. Let’s go ahead and remove the first container by running docker rm -f app-container. The -f option will force the removal of the container even if it’s still running. Now that we have the old container removed, let’s run ‘docker run -d -p 8000:8000 –name app-container app’ again. Check your browser again to see if the app is loading. If everything looks fine, then we are ready to move into the actual deployment with Jenkins.

On your browser, head over to our Jenkins URL http://localhost:8080. Create a new job by clicking the create new jobs link.

Jenkins New Job

Jenkins New Job

Enter a descriptive name and click ‘Freestyle project’ then click Ok. Under General, check GitHub project and enter your GitHub url for the Project url. Under Source Code Management check Git and enter the same url for the Repository URL.

Jenkins Job Configuration

Jenkins Job Configuration

Under Build Triggers, check Build when a change is pushed to GitHub. Under Build, click Add build step and select Execute shell.

Jenkins Job Config

Jenkins Job Config

In the command textbox, enter the following:

Click Save.

Back to your GitHub repo, we need to integrate our Jenkins installation into our repo by adding a webhook to it. Click Settings under your GitHub repo, click Integration & Services on the left side, under Services click Add service and select Jenkins (GitHub plugin). Under Jenkins hook url, enter your Jenkins url like http://x.x.x.x:8080/github-webook/ then click Add service. Please make sure your Jenkins installation is visible on the internet. You can easily do this by forwarding port 8080 from your router to your machine running the VM. What this does is whenever a push is made to your GitHub repo, GitHub will then send a webhook to your Jenkins installation which in turn will run the build we setup above.

Lastly, on your terminal, commit and push your changes to your repo. Watch the Jenkins job we just created and it should show a building status on Build History after a while. Click the build number and then click Console Output. You should be able to see what’s happening to your build through the console output. Once the build finishes, check if your app was deployed by running accessing http://localhost:8000. You can also check if the container is running by running docker ps inside your VM. If the app container is running, then congratulations! You just made your own GitHub-Jenkins CI setup using Vagrant and Docker!

To get the complete source code used in this project, head over to https://github.com/iambryancs/git-jenkins-ci-cd.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: