Jenkins – The Juggernaut of Continuous Delivery

09 May 2016

Los Hermanos Jenkins

Today, continuous delivery (CD) is en-vogue. As companies adopt CD, they are faced with a choice on the tools to establish their delivery pipelines. Jenkins is often the obvious choice and best of all it is likely used by the engineering division already. This blog looks at various data points to paint a picture of the pervasiveness of Jenkins for CD.

Blog When I started at CloudBees a half a dozen years ago, CD wasn’t

in the lexicon – in fact I named “Jenkins Pipelines” feature to be “Workflow” – pipelines is much better! Asking customers about their Dev-to-Production software delivery strategy was often met with a quizzical look.

The four-minute-mile barrier of software delivery was delivering software maybe once-a-month and that barrier has long been broken. Fabled stories of Amazon, Etsy and FaceBook delivering software multiple times a day have set a new bar that every company aspires to.

Google trends shows a continuous (pun intended) rise in the interest of CD. Trends

Since data science is the hottest trend in the market now, I have used data science to provide for a relative scale.Data science

It has been clear that Jenkins is the de-facto CI leader in the market. The ZeroTurnAround survey shows that Jenkins is most adopted CI server in the market with 70% of the market share.

ZeroTurnaround

(source: Zeroturnaround)

Active (ones that report active usage) Jenkins installations count is 127k and growing. Active Installations

(source: Jenkins stats site)

My guess is that non-reporting installations must be 2-4 times the active installations. Typically, a Jenkins installation supports anywhere between 5-100 people (dev, qa etc) – conservatively this could mean anywhere from 1M+ people using Jenkins (counting 10 developers per active installation).

Let’s look at data from Jenkins Survey in 2013 and 2015 to see some trends.

Jenkins for Operations In the 2 years between the surveys, the

usage of Jenkins for operations activities has grown by 9%.

Jenkins for  Operations

(source: Jenkins 2013 survey) Jenkins Jobs (source: Jenkins 2015 survey)

Jenkins for Deployment The story for deployment is even better with

60% of Jenkins users using the tool for deployment. Note: this number is likely higher as in some cases activities under “release, operations” overlap into the deployment zone. About 70% of those doing deployment are either doing continuous delivery (manual flag to deploy to production) or continuous deployment (auto-deploy).

Deployment

(source: Jenkins 2015 survey infographic)

If we take the 127k active installation number and apply it here (70% of 127k)- we have about 88k installations using Jenkins for deployment. Is Jenkins the #1 CD tool in the market? Looks like it! Can other tools make this claim – doubt it!

83% of deployments happen multiple times a week! Talk about breaking the 4 minutes mile barrier! Delivering is no longer stressful!

Multiple deployments

Building modern pipelines with Docker

Last year, the community churned out a number of plugins to help build pipelines with Docker. It is no surprise to now that Jenkins with Docker is a common use case. In other story, Jenkins is the most popular Docker image on Docker Hub with 5M+ (yes million) pulls.

Increase in the number of plugins

A key reason for Jenkins’ success has been the number of plugins that connect into existing tools. After all, if an organisation brings in a tool to do CD – it has to work with existing tools in the market.

Jenkins has about 1100+ plugins that connect to almost any tool out there. The number of installations of these plugins is north of 4.5M (yes million – it isn’t a typo). Usage

No wonder, the actual usage of Jenkins within an organisation keeps increasing with 89% respondents saying that their usage in Jenkins increased.

Actual usage

Mission Critical Jenkins

Given that an organisations’ entire software delivery chain is dependent on Jenkins, it is no surprise Jenkins is considered mission critical. Between 2013 and today the growth is about 9%. 92% of respondents say that Jenkins is mission critical – not a surprise, given that at-least 50% of developers in organisation are using Jenkins. A colleague asked a question in talk recently “What would happen if Jenkins goes down?”, only to be met with nervous laughter.

Mission critical

A testament to Jenkins is that 89% of teams are satisfied with Jenkins. Satisfaction

This satisfaction shows up in numerous awards!

Awards

Increase in the number of jobs – Jenkins jobs

In the last 3 years, the number of jobs created in Jenkins has grown almost 3 times from ~2.25M to 7M. Number of jobs

Increase in the number of jobs – human jobs

With the increase in Jenkins usage, its usage in CD, there is no surprise that people in the “DevOps” role using Jenkins has more than doubled to 48%. Number of human jobs

This is somewhat borne in the job postings in the market as well. There seems to be a growth in the specialised role that is focused on continuous delivery. There is a corresponding growth in Jenkins jobs – doubling in the last 2 years!

Job Role

Job Trend

To make sense of the data, I added data scientist job in there for fun 😃 and was more than presently surprised to see that there are 3x jobs in continuous delivery. Job trend 2

So what does this mean for you?

The Jenkins community has worked in the last two years to build the Jenkins Pipeline feature that brings native support for Pipelines. So if you are considering walking the continuous delivery path, ask your engineers to look at Jenkins Pipelines – most likely they are already using Jenkins and Pipelines is just a plugin away. Join the 79% of Jenkins users who are going to move to pipelines in the next 12 months.

Pipeline plugin

I have given an overview in an earlier blog and more recently the Jenkins community did a virtual conference that goes into details here. These should be good starts.

Summary

Unicorn companies have proven to the world that delivering software continuously is possible. Regular companies are now well on the path of delivering software continuously. A significant number of Jenkins users are now using it to deliver software continuously. Given, Jenkins’ large reach, it is not a stretch to say that it is the #1 tool in the market to do CD.

Opinions here are all mine and do not reflect that of my employers.

Reference

Jenkins 2.0: Makes it easy for organisations to deliver software continuously

28 Apr 2016

I have spent the last 3 years of my life helping Jenkins claim its place upfront and central in the Continuous Delivery market. The journey first started by providing first class support for CD pipelines through Jenkins Pipelines and has culminated in the first major release of Jenkins in 10 years. I am incredibly privileged to have helped Kohsuke Kawaguchi in this journey. The following is a blog post that I put out on cloudbees.com for the release.

Jenkins 2.0 is generally available now. It is the first major release of Jenkins in 10 years after about 700 weekly releases. It is indeed a major milestone in the Jenkins project. The primary focus of the release is Pipelines – a means to helping organisations deliver better software faster.

Why should I care?

A fair enough question so I will wear a different “I” hat and answer why Jenkins 2.0 is relevant to you.

Role: Engineering Manager

Key Issue: My team delivers an application from development to production – what does Jenkins 2.0 bring to the table?

Pipelines: An easy to use DSL to model, view and optimise your application delivery value stream that can significantly improve the pace of delivery for your applications.

Jenkins 2.0 natively supports the notion of pipelines and helps to model your delivery pipeline through a DSL that is checked into the source code repository (Jenkinsfile). This file can then be version controlled and shared with other teams.

If your team was using multiple (existing) Jenkins plugins to model your pipeline, Pipelines greatly simplifies expressing the delivery pipeline, maintaining and managing it.

Pipeline stage view helps your team analyse the performance of the delivery pipeline and hone in on problematic stages in the delivery chain. Pipeline Stage View

Pipelines support GitHub and BitBucket organisations and pull requests. Point Jenkins to a GitHub organisation that has a Jenkinsfile checked in, Jenkins will automatically create a job, run and manage it – very useful when onboarding new teams.

Role: Shared Services Engineering Manager, VP of Engineering, Tools Group Manager

Key Issues:

  • My team is responsible for minimizing the impact of downtimes for various teams that depend on the tools supplied by my team.
  • My team is responsible for fostering best practices across engineering groups delivering applications.

Pipelines survive infrastructure outages; common delivery patterns can be stored in global libraries that are shared across various teams.

Pipelines are durable and resumable. Thus, if you support an engineering team whose application delivery spans multiple days, you can support them with the confidence to know that pipelines has inbuilt resilience to survive infrastructure failure. The pipeline job continues running even if the master fails. If you use CloudBees Jenkins Platform – your job can resume from the last checkpoint on executor failures.

Pipelines follow the DRY principle – your team can capture common deliverability patterns (“deploy to Tomcat”) into functions that can be invoked by teams that you support. Thus, your team can share best practices within the organisation.

Role: An existing Jenkins user

Key Issue: Any usability improvements?

Besides Pipelines, Jenkins 2.0 brings in a number of improvements for usability. Jenkins 1.x configuration page becomes overloaded with configuration choices introduced by plugins. Improvements to the configuration management has been an area of focus for Jenkins 2.0.

Config UI

A new startup wizard helps newbies get to a running installation quickly with a pre-configured set of plugins as well.

Role: Jenkins administrator Key Issue: Will an upgrade break an existing installation?

Jenkins 2.0 is backward compatible and a drop-in replacement. So upgrade without worries.

So go ahead – bring in Jenkins 2.0 and earn good karma within your organisation

Resources: