node { // <1>
stage('Build') { // <2>
/* .. snip .. */
}
stage('Test') {
/* .. snip .. */
}
stage('Deploy') {
/* .. snip .. */
}
}
This section builds on the information covered in Getting Started,
and introduces more useful steps, common patterns, and demonstrates some
non-trivial Jenkinsfile examples.
Creating a Jenkinsfile, which is checked into source control
[1],
provides a number of immediate benefits:
Code review/iteration on the Pipeline
Audit trail for the Pipeline
Single source of truth [2] for the Pipeline, which can be viewed and edited by multiple members of the project.
While the syntax for defining a Pipeline, either in the web UI or with a
Jenkinsfile, is the same, it’s generally considered best practice to define
the Pipeline in a Jenkinsfile and check that in to source control.
As discussed in the Getting Started
section, a Jenkinsfile is a text file that contains the definition of a
Jenkins Pipeline and is checked into source control. Consider the following
Pipeline which implements a basic three-stage continuous delivery pipeline.
node { // <1>
stage('Build') { // <2>
/* .. snip .. */
}
stage('Test') {
/* .. snip .. */
}
stage('Deploy') {
/* .. snip .. */
}
}| 1 | node allocates an executor and workspace in the Jenkins environment. |
| 2 | stage describes distinct parts of the Pipeline for better visualization of progress/status. |
Not all Pipelines will have these same three stages, but this is a good continuous delivery starting point to define them for most projects. The sections below will demonstrate the creation and execution of a simple Pipeline in a test installation of Jenkins.
|
It is assumed that there is already a source control repository set up for the project and a Pipeline has been defined in Jenkins following these instructions. |
Using a text editor, ideally one which supports
Groovy
syntax highlighting, create a new Jenkinsfile in the root directory of the
project.
In the example above, node is a crucial first step as it allocates an
executor and workspace for the Pipeline. In essence, without node, a Pipeline
cannot do any work! From within node, the first order of business will be to
checkout the source code for this project. Since the Jenkinsfile is being
pulled directly from source control, Pipeline provides a quick and easy way to
access the right revision of the source code
node {
checkout scm // <1>
/* .. snip .. */
}| 1 | The checkout step will checkout code from source control; scm is a
special variable which instructs the checkout step to clone the specific
revision which triggered this Pipeline run. |
For many projects the beginning of "work" in the Pipeline would be the "build"
stage. Typically this stage of the Pipeline will be where source code is
assembled, compiled, or packaged. The Jenkinsfile is not a replacement for an
existing build tool such as GNU/Make, Maven, Gradle, etc, but rather can be
viewed as a glue layer to bind the multiple phases of a project’s development
lifecycle (build, test, deploy, etc) together.
Jenkins has a number of plugins for invoking practically any build tool in
general use, but this example will simply invoke make from a shell step
(sh). The sh step assumes the system is Unix/Linux-based, for
Windows-based systems the bat could be used instead.
node {
/* .. snip .. */
stage('Build') {
sh 'make' // <1>
archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true // <2>
}
/* .. snip .. */
}| 1 | The sh step invokes the make command and will only continue if a
zero exit code is returned by the command. Any non-zero exit code will fail the
Pipeline. |
| 2 | archiveArtifacts captures the files built matching the include pattern
(**/target/*.jar) and saves them to the Jenkins master for later retrieval. |
|
Archiving artifacts is not a substitute for using external artifact repositories such as Artifactory or Nexus and should be considered only for basic reporting and file archival. |
Running automated tests is a crucial component of any successful continuous
delivery process. As such, Jenkins has a number of test recording, reporting,
and visualization facilities provided by a
number of plugins.
At a fundamental level, when there are test failures, it is useful to have
Jenkins record the failures for reporting and visualization in the web UI. The
example below uses the junit step, provided by the
JUnit plugin.
In the example below, if tests fail, the Pipeline is marked "unstable", as denoted by a yellow ball in the web UI. Based on the recorded test reports, Jenkins can also provide historical trend analysis and visualization.
node {
/* .. snip .. */
stage('Test') {
/* `make check` returns non-zero on test failures,
* using `true` to allow the Pipeline to continue nonetheless
*/
sh 'make check || true' // <1>
junit '**/target/*.xml' // <2>
}
/* .. snip .. */
}| 1 | Using an inline shell conditional (sh 'make || true') ensures that the
sh step always sees a zero exit code, giving the junit step the opportunity
to capture and process the test reports. Alternative approaches to this are
covered in more detail in the Handling Failures section below. |
| 2 | junit captures and associates the JUnit XML files matching the inclusion
pattern (**/target/*.xml). |
Deployment can imply a variety of steps, depending on the project or organization requirements, and may be anything from publishing built artifacts to an Artifactory server, to pushing code to a production system.
At this stage of the example Pipeline, both the "Build" and "Test" stages have successfully executed. In essense, the "Deploy" stage will only execute assuming previous stages completed successfully, otherwise the Pipeline would have exited early.
node {
/* .. snip .. */
stage('Deploy') {
if (currentBuild.result == 'SUCCESS') { // <1>
sh 'make publish'
}
}
/* .. snip .. */
}| 1 | Accessing the currentBuild.result variable allows the Pipeline to
determine if there were any test failures. In which case, the value would be
UNSTABLE. |
Assuming everything has executed successfully in the example Jenkins Pipeline, each successful Pipeline run will have associated build artifacts archived, test results reported upon and the full console output all in Jenkins.
A Scripted Pipeline can include conditional tests (shown above), loops, try/catch/finally blocks and even functions. The next section will cover this advanced Scripted Pipeline syntax in more detail.
Scripted Pipeline is a domain-specific language [3] based on Groovy, most Groovy syntax can be used in Scripted Pipeline without modification.
Groovy’s "String" interpolation support can be confusing to many newcomers to the language. While Groovy supports declaring a string with either single quotes, or double quotes, for example:
def singlyQuoted = 'Hello'
def doublyQuoted = "World"Only the latter string will support the dollar-sign ($) based string
interpolation, for example:
def username = 'Jenkins'
echo 'Hello Mr. ${username}'
echo "I said, Hello Mr. ${username}"Would result in:
Hello Mr. ${username}
I said, Hello Mr. JenkinsUnderstanding how to use Groovy’s string interpolation is vital for using some of Scripted Pipeline 's more advanced features.
Jenkins Pipeline exposes environment variables via the global variable env,
which is available from anywhere within a Jenkinsfile. The full list of
environment variables accessible from within Jenkins Pipeline is documented at
localhost:8080/pipeline-syntax/globals#env,
assuming a Jenkins master is running on localhost:8080, and includes:
The current build ID, identical to BUILD_NUMBER for builds created in Jenkins versions 1.597+
Name of the project of this build, such as "foo" or "foo/bar".
Full URL of Jenkins, such as example.com:port/jenkins/ (NOTE: only available if Jenkins URL set in "System Configuration")
Referencing or using these environment variables can be accomplished like accessing any key in a Groovy Map, for example:
node {
echo "Running ${env.BUILD_ID} on ${env.JENKINS_URL}"
}Setting an environment variable within a Jenkins Pipeline can be done with the
withEnv step, which allows overriding specified environment variables for a
given block of Scripted Pipeline, for example:
node {
/* .. snip .. */
withEnv(["PATH+MAVEN=${tool 'M3'}/bin"]) {
sh 'mvn -B verify'
}
}If you configured your pipeline to accept parameters using the Build with Parameters option, those parameters are accessible as Groovy variables of the same name.
Assuming that a String parameter named "Greeting" has been configured for the
Pipeline project in the web UI, a Jenkinsfile can access that parameter via
$Greeting:
node {
echo "${Greeting} World!"
}Scripted Pipeline relies on Groovy’s built-in try/catch/finally semantics
for handling failures during execution of the Pipeline.
In the Test example above, the sh step was modified to never return a
non-zero exit code (sh 'make check || true'). This approach, while valid,
means the following stages need to check currentBuild.result to know if
there has been a test failure or not.
An alternative way of handling this, which preserves the early-exit behavior of
failures in Pipeline, while still giving junit the chance to capture test
reports, is to use a series of try/finally blocks:
node {
/* .. snip .. */
stage('Test') {
try {
sh 'make check'
}
finally {
junit '**/target/*.xml'
}
}
/* .. snip .. */
}In all previous uses of the node step, it has been used without any
arguments. This means Jenkins will allocate an executor wherever one is
available. The node step can take an optional "label" parameter, which is
helpful for more advanced use-cases such as executing builds/tests across
multiple platforms.
In the example below, the "Build" stage will be performed on one node and the built results will be reused on two different nodes, labelled "linux" and "windows" respectively, during the "Test" stage.
stage('Build') {
node {
checkout scm
sh 'make'
stash includes: '**/target/*.jar', name: 'app' // <1>
}
}
stage('Test') {
node('linux') { // <2>
checkout scm
try {
unstash 'app' // <3>
sh 'make check'
}
finally {
junit '**/target/*.xml'
}
}
node('windows') {
checkout scm
try {
unstash 'app'
bat 'make check' // <4>
}
finally {
junit '**/target/*.xml'
}
}
}| 1 | The stash step allows capturing files matching an inclusion pattern
(**/target/*.jar) for reuse within the same Pipeline. Once the Pipeline has
completed its execution, stashed files are deleted from the Jenkins master. |
| 2 | The optional parameter to node allows for any valid Jenkins label
expression. Consult the inline help for node in the Snippet Generator for more details. |
| 3 | unstash will retrieve the named "stash" from the Jenkins master into the
Pipeline’s current workspace. |
| 4 | The bat script allows for executing batch scripts on Windows-based
platforms. |
The example in the section above runs tests across two
different platforms in a linear series. In practice, if the make check
execution takes 30 minutes to complete, the "Test" stage would now take 60
minutes to complete!
Fortunately, Pipeline has built-in functionality for executing portions of
Scripted Pipeline in parallel, implemented in the aptly named parallel step.
Refactoring the example above to use the parallel step:
stage('Build') {
/* .. snip .. */
}
stage('Test') {
parallel linux: {
node('linux') {
checkout scm
try {
unstash 'app'
sh 'make check'
}
finally {
junit '**/target/*.xml'
}
}
},
windows: {
node('windows') {
/* .. snip .. */
}
}
}Instead of executing the tests on the "linux" and "windows" labelled nodes in series, they will now execute in parallel assuming the requisite capacity exists in the Jenkins environment.
Groovy allows parentheses around function arguments to be omitted.
Many Pipeline steps also use the named-parameter syntax as a shorthand for
creating a Map in Groovy, which uses the syntax [key1: value1, key2: value2].
Making statements like the following functionally equivalent:
git url: 'git://example.com/amazing-project.git', branch: 'master'
git([url: 'git://example.com/amazing-project.git', branch: 'master'])For convenience, when calling steps taking only one parameter (or only one mandatory parameter), the parameter name may be omitted, for example:
sh 'echo hello' /* short form */
sh([script: 'echo hello']) /* long form */