Probably two of the best tools for creating jenkins jobs programmatically are the Jenkins Job DSL Plugin and the Jenkins Job Builder. In this two part post contains a short introduction and comparison of these two tools. This post includes an introduction to the Jenkins Job Builder.
All the examples used in this introduction are available here along with a short description on how to run the examples.
The Jenkins Job Builder comes from Open Stacks Infrastructure components. It uses job definitions in YAML format to generate jenkins job xml structure and upload them to a jenkins server. A simple job looks like this:
|
1 2 3 4 5 6 |
- job: name: Very basic freestyle job project-type: freestyle description: 'Just contais a very basic jenkins job that echos "Hello world"' builders: - shell: echo 'Hello world!' |
As you can see it is very easy to read and can be edited with a simple text editor (although yaml highlighting would be a plus).
Before you can convert this yaml to a jenkins job you have to install the job builder as described next. There is also an excellent and exhausting documentation available here.
The first step is to clone the git repository:
|
1 |
git clone https://github.com/openstack-infra/jenkins-job-builder |
ant then install it:
|
1 |
sudo python setup.py install |
This instructions assumes that you have the python package manager “pip” installed. If not you can install it using yum:
|
1 |
yum install python-pip |
When I first ran the job builder on my machine, I got an the error message:
No module named pbr.version
And I had to install the python package “Python Build Reasonableness” from openstack:
|
1 |
pip install pbr |
No you can run the jenkins job builder like this to see the xml actually generated:
|
1 |
jenkins-jobs test ./sample1.yaml |
The xml output looks something like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
<project> <actions/> <description>Just contais a very basic jenkins job that echos "Hello world"<!-- Managed by Jenkins Job Builder --></description> <keepDependencies>false</keepDependencies> <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> <concurrentBuild>false</concurrentBuild> <canRoam>true</canRoam> <properties/> <scm class="hudson.scm.NullSCM"/> <builders> <hudson.tasks.Shell> <command>echo 'Hello world!'</command> </hudson.tasks.Shell> </builders> <publishers/> <buildWrappers/> </project> |
To actually upload the job to a jenkins server you first have to create a configuration file “jenkins_jobs.ini”. You can either place the file in “/etc/jenkins_jobs” or specify the exact location to the jenkins job builder using the “–conf” parameter.
The following command lets you upload the job:
|
1 |
jenkins-jobs update ./sample1.yaml |
You can then take a look at the resulting console through the jenkins UI:
This wasn’t to hard. We will no delve into more details of the job builder especially how to generate jobs from templates.
Reuse and templating
When having a large number of jobs to create the obvious need for templating and reusing common parts in job definitions arises. The Job Builder offers three ways for that:
Not that if you scatter the definitions that make up your jobs over multiple files you have to tell the jenkins-jobs program to include all the directories and files that contain parts of the job definition.
For example if you define your defaults in a file defaults.xml and the jobs that uses them in a file job.xml, jenkins-jobs need to be passed both files as a parameter. Or the parent directory. How to do that is described here in detail.
Now back to reusing parts of your jobs: Default values for jobs may be put in separate files and reused in job definition files. The following file contains a definition for default values:
|
1 2 3 4 5 6 7 8 |
- defaults: name: global description: 'Do not edit this job through the web!' properties: - authorization: anonymous: - job-read - job-discover |
And these default values may be reused as follows in job definitions:
|
1 2 3 4 5 6 7 |
- job: defaults: global name: Job with script include description: Contains an include of a script project-type: freestyle builders: - shell: !include-raw scripts/helloworld.sh |
As the name implies, default are fixed values. In order to parametrized job definitions, templates may be used. Templates follow the same structure as job definitions with the exception that you may use parameters:
|
1 2 3 |
- job-template: name: '{name}-build' description: 'Description of the job named {name}-build' |
The above yaml file defines a template which expects a parameter name. Note that a template must contain a parameter in the name attribute. Otherwise all the job derived from the template would have the same name.
To use templates a project needs to be defined, in which the template is used:
|
1 2 3 4 5 6 7 8 9 |
- project: name: example1 jobs: - '{name}-build' - project: name: example2 jobs: - '{name}-build' |
The example from this file contains the project definitions which use the template “{name}-build” and use two different names. As a result two jobs are generated.
If you do not intent to reuse whole job definitions but only certain parts, macros might be the right choice. Macros let you externalize and parametrize certain sections of a job:
|
1 2 3 4 |
- builder: name: add builders: - shell: "echo Adding {number}" |
The above example defines a macro named “add” that adds a shell build step with a parameter. To use this template, it has to be inserted in the job definition:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
- job: name: A job that uses marcors project-type: freestyle defaults: global description: 'Contains marco definition and usage' builders: - add: number: "one" - add: number: "two" - add: number: "three" |
In the example above three build steps are inserted with different values for the number.
Apart from these three mechanism, there is the possibility to use yuml anchors and aliases to include arbitrary pieces of a yuml file at another position. See here and here for more information. If those two approaches actually increase the readability of the job definitions remains questioned.
Finally the Jenkins Job Builder supports almost all popular jenkins plugins. But in rare case where a plugin is not supported you may actually extend the job builder, as explained here.
Conclusion
The jenkins job builder is easy to setup and uses plain text files for job definitions, which lets you get up and running quiet easily. At the same time it offers powerful mechanisms for templating, reuse and programtically generated jobs. Which is important in bigger projects and companies.
And last but not lease it does not require any plugin to be installed in jenkins.
In the next post of this series we will be looking into the Jenkins Job DSL Plugin which overs DSL and groovy based job definitions. That part also contains a comparison between the two tools.
I wrote a blog post on some of the more advanced things you can do with the Job DSL. I talk about the following topics:
– Factory/Builder pattern using a class library
– Configure Block & Extending the DSL
– Environment Based Configuration
– User management in Code
– Shared Data from Configuration Management
You can check it out here: https://blog.thesparktree.com/you-dont-know-jenkins-part-2