Home/Blog/Test Pipeline Failures Before They Hit CI
tutorialtestingdevops

Test Pipeline Failures Before They Hit CI

Plumber's fault injection lets you force any stage to fail and test your post{failure{}} handlers locally — before a broken deploy lands in production at 2am.

AO

Abi O.

Creator

8 May 20254 min read

Nothing tests your incident response faster than a 2am deploy failure with broken post { failure {} } handlers. The Slack notification never fires, the ticket never opens, and nobody gets paged until someone checks Jenkins manually.

Plumber has a feature specifically for this: fault injection.

How fault injection works

In the Plumber timeline, right-click any stage and you'll see two options:

  • Force Fail — this stage always exits with code 1, triggering your failure handlers
  • Set failure rate — enter a percentage (e.g. 30%) and Plumber will randomly fail this stage, simulating a flaky test or intermittent network call

With Force Fail enabled on your Deploy stage, hit Run. Your full pipeline executes, reaches Deploy, fails, and then Plumber runs every post { failure {} } block — including nested ones — with full log output.

What you can validate

  • Notification handlers (Slack, email, PagerDuty via shell steps)
  • Cleanup steps — do your containers actually stop on failure?
  • Artifact archiving — does your test report still get saved even when a later stage fails?
  • Parallel failure propagation — does failing one branch of a parallel {} correctly cancel siblings?

A real example

Say you have this pipeline:

groovycode
pipeline {
  agent any
  stages {
    stage('Deploy') {
      steps {
        sh './deploy.sh production'
      }
    }
  }
  post {
    failure {
      sh 'curl -X POST $SLACK_WEBHOOK -d "{\"text\":\"Deploy failed!\"}"'
    }
    always {
      sh './cleanup.sh'
    }
  }
}

Force-fail the Deploy stage. Plumber will run the failure {} block and the always {} block in sequence. You'll see immediately whether your webhook URL is correct, whether your cleanup script handles a missing container gracefully, and whether your environment variables are set.

Flaky tests are a first-class citizen

The random failure rate option is useful for simulating flaky test suites. Set it to 40% on your test stage and run the pipeline ten times. You'll quickly see whether your retry logic actually works, or whether it silently succeeds on re-runs without surfacing the underlying instability.

If your post-failure handlers have never actually run, you don't know if they work.

Give it a try. Your on-call rotation will thank you.