Pipeline as Code revolutionized how continuous delivery pipelines are defined in Jenkins by checking in the Pipeline as a ‘Jenkinsfile’ in your repository instead of storing the definition locally in Jenkins. This becomes especially useful when leveraging the direct integrations Jenkins has with Github and Bitbucket . In this case, Jenkins will scan the entire Organization for repositories containing Jenkinsfiles and then create the associated pipelines in Jenkins automatically.
The Jenkinsfile approach is great for many use cases but many times larger organizations would like to use Pipeline as Code while still setting certain standards to be used throughout the organization. Pipeline as Code allows free reign to developers to create their own Jenkinsfiles without any regard to corporate standards or practices. This may be fine with a small implementation, but as the as the number of projects and repos increase, there is a higher chance of teams deviating from the best practices developed by the shared services team. Additionally, there are many instances where you may just want to ensure that certain commands run before or after the general Pipeline, such as requiring a cleanup to occur after any build runs in the Organization. The Custom Marker File feature in the CloudBees Jenkins Platform was created to address some of these concerns.
As blogged about earlier , Custom Marker Files allow you to associate repositories in your SCM that have a given identifier with a generic Pipeline. For example, instead of having the same Jenkinsfile defined in every Java project, Custom Marker Files will instead allow you to define one Pipeline that should be used by all repositories that have a ‘pom.xml’ file in it (with pom.xml being the identifier that lets you know that this is a Java project). This functions as an easy way to get new teams onboarded because once they create a new Java Project in Github/Bitbucket, Jenkins will start building the project without the need of creating a new Jenkinsfile.
In many cases, fully templatizing the Jenkinsfile for the entire organization as described above may be too restrictive, which is why Custom Marker Files also allow you to set standards while still giving teams the flexibility to create their own Jenkinsfiles. Let’s go through an example of how this would work in a Github Organization (works with Bitbucket Teams as well). Make sure to have updated versions of the CloudBees Pipeline: Templates plugin.
- Click New Item -> Github Organization and give it a name to create a new Github Organization
- In the Configuration page, enter in the Github Organization and Scan Credentials
- In the Project Recognizers section
- Delete the option “Pipeline Jenkinsfile”
- Click “Add” and select “Custom script”
- Enter in the information as shown below
- Click Save to start scanning your organization
Notice that in this case the Marker File is set to a Jenkinsfile. This means that any repository or branch in the Organization with a Jenkinsfile will be detected. The Pipeline section then defines the Pipeline that will be used for all projects containing the ‘Jenkinsfile’ Marker File. Let’s analyze this Pipeline script.
- Lines 1-3 prepends a ‘preflight checks’ stage that allows custom actions such as setting up the build environment before the main Jenkinsfile Pipeline.
- Line 5 uses the ‘readTrusted’ function to read the Jenkinsfile file into a variable. readTrusted allows you to read files from the project that is being checked out without the requirement of being on a node.
- Line 6 then actually runs the Jenkinsfile pipeline by using the ‘evaluate ’ function. ‘evaluate’ is similar to ‘load’ but does not require to be run on a ‘node’ as well.
- Lines 8-10 appends a ‘postflight cleanup’ stage where cleanup commands can be run after the pipeline has completed.
Looking at the Stage View above, you can see that the main Pipeline ran with the ‘preflight checks’ stage prepended and the ‘postflight cleanup’ stage appended.
This method of reading in the Jenkinsfile for Custom Marker Files can be useful for requiring additional actions to be run across the organization. The example in this blog is very simple but this can also be a good starting place in the future to analyze the Jenkinsfile that developers create and ensure that it meets certain standards. An example of this would be to read in the Jenkinsfile into a variable (look at Line 5) and parse that variable to make sure that certain functions are used or that it adheres to a certain schema. With all of these different options, Custom Marker Files in the CloudBees Jenkins Platform help you strike a balance between standardizing and providing adequate freedom for your developers.
Isaac Cohen is a Solutions Architect at CloudBees , enabling customers to achieve their DevOps \ Continuous Delivery goals and making the overall SDLC as efficient and transparent as possible. Isaac has extensive knowledge of Jenkins and its various tool integrations as well as hands-on experience leading DevOps teams for large enterprises.