Jenkins Pipeline As Code

Baptiste Mathus
@bmathus

About Me

def BaptisteMathus =
    people.filter(p -> p.isHacker())
          .filter(p -> p.isOpenSourceFan())
          .filter(p -> p.lovesCommunities())
          .filter(p -> p.isCommitterOn("Jenkins"))
          .filter(p -> p.isCommitterOn("MojoHaus"))
          .filter(p -> p.doesRidiculousJokes())
          .uniqueResult()
batmat about

#jenkins #java #docker #maven #automation #devops #passion

Agenda

  • Quick History Of Jenkins
  • Build Jenkins!
  • Build Jenkins, again! Quicker.

A bit of history

2005 : Hudson created by Kohsuke Kawaguchi

2011 : Oracle forks the project.

The community renames the project to Jenkins and goes away

2016 : Jenkins Project Still Vibrant

jenkins infographics

Let’s build…​ Jenkins

Typically. You would do:

freestyle 1
freestyle 2
  • Deploy to test environment? User Acceptance Testing?
freestyle 3

Or, create a bunch of jobs.

Wait. What if I want to

  • Wait for user input?

  • Parallelize some of the steps?

    • fail-fast?

  • Deploy to a testing environment

    • after successful standard tests,
    • but before UA Testing is finished?
    • If UAT fails, rollback to previous version?

  • And more…​

Pipeline: From Code To Deployed App

pipeline concept

But what about maintenance?

  • Versioning?
  • Readability?
  • Robustness?

Everything As Code

infra as code tools

When everything is code, then versioning comes (almost) for free.

Also, rollback.

Go faster!

Deliver As Fast As Possible!

speed

What is a commit that is not running in production?

A Commit Not Yet In Production Is Stock!

The Jenkins Pipeline Plugin(s)

  • Scripted — Domain Specific Language
  • Pausable
  • Extensible
  • Fist public beta release: June 2014

So, now. Let’s get back to building Jenkins

node ('label') {

  git "https://github.com/jenkinsci/jenkins.git"

  // triggers the tool install as usual
  def java  = tool 'jdk-1.8.0'
  def maven = tool 'maven-3'

  withEnv(["JAVA_HOME=$java",
           "PATH+MAVEN=$maven/bin:
           ${env.JAVA_HOME}/bin"]) {

    sh "mvn clean deploy ..."

  }
}

Question to the user?

input message: "Deploy to prod?"
input message: "Deploy to prod?",
  parameters:[[$class:'StringParameterDefinition',
               defaultValue: 'NOW',
               description: '...',
               name: 'hour'
               ]]
}

Multi-nodes

stash: To pass things around

stash includes: '**/*.jar,war/target/*.war',
      name: 'bucket'

...

unstash 'bucket'

Parallelism

parallel core: {

},       tests: {

}

Fail fast?

parallel core: {

},       tests: {

}, failFast: true

parallel actually takes a Map

def execs = [:] // creates empty map
for (int i=0;i<10;++i) {

    def index = i
    execs [i] = {

        node {
            echo "Executing for $index"
            sleep 10
        }
    }
}
execs.failFast = true
parallel execs
parallel execs example

Visualization?

stage lets you define logical steps of your pipeline.

Today:

stage "Checkout"
...
...
stage "Prebuild"
...
...

Soon:

stage ("Prebuild") {
  ...
  ...
}

The Stage View Plugin

stage view

But then again…​

  • What about versioning?
  • How to handle many [feature] Branches?

Jenkinsfile!

Multibranch Pipeline

  • Define Pipeline in a Jenkinsfile in the repo and create job:
multibranch

Editing facilities?

  • Snippet Generator
snippet generator
  • IntelliJ GDSL: bringing autocompletion to DSL
  • More To Come

Demo!

demo

Infrastructure:

jenkins docker swarm

Rationale: from linear to (naively) parallel

  1. prebuild things
  2. stash them
  3. split tests per starting letter
  4. spawn them on one node

Code

Prebuild:

node ('demo') {
  stage: "Clone"
  git 'https://github.com/jenkinsci/jenkins.git'

  stage "Install Maven"

  stage "Prebuild"
  withEnv(["PATH+MAVEN=${tool 'maven-3'}/bin"]) {
    sh "mvn ... package -DskipTests..."
  }
  stash includes: '**', name: 'prebuilt'
}

Prepare tests:

def tests = [:]
for (char letter='A';letter<'Z';++letter) {
  tests["$letter tests"] = {
    node ('demo') {
      unstash 'prebuilt'

      withEnv(["PATH+MAVEN=${ tool 'maven-3' }/bin"]) {
        sh "mvn ... package '-Dtest=$letter*Test'"
      }
    }
  }
}

Gotchas

  • C-style loops only - no functional Groovy yet (JENKINS-26481)
  • Script approval can be cumbersome (aka whitelisting)
  • Existing plugins support — Though quickly improving

The Future

  • Still early days, but already very usable
  • More and more real world feedback
  • Visual Designer
  • IDE support (Eclipse anyone?)
  • BlueOcean! Developer Experience Revisited

Come to us. We have cookies.

Thanks

do logo
mipih logo

For the Cloud Credits

We’re hiring!
econ feedback2

References

← →

/