Skip to main content

Stop Waiting for CI. Start Getting Productive.

30-minute Vendor Talk

CI should validate a change, not be the primary environment for debugging failures.

Virtual Pass session

Timetable

3:45 p.m. – 4:30 p.m. Wednesday 18th

Room

Room E1 - Track 4: Talks

Continuous Integration/Continuous Delivery (CI/CD) Test Automation Testing Tools

Audience

Engineers, Tech Leads, Project Leads, Managers

Key-Learnings

  • Closely coupling integration tests to CI makes failures unnecessarily difficult to reproduce.
  • Reproducible integration test failures enable efficient debugging.
  • If done right, the push–wait–debug cycle can be replaced with fast local iteration.

Almost every software engineer has experienced the frustration of debugging integration tests through repeated CI runs. The test fails in CI, and every attempted fix requires another commit, another push, and another wait for feedback. The root cause is often simple: the integration test cannot be reproduced exactly outside the CI environment.

In this talk, we argue that this is not an inevitable part of software development, but a consequence of treating CI as a special environment. If developers can execute the exact same integration test in the exact same environment on their own machine, they can reproduce, analyze, and fix failures without ever pushing a change.

We demonstrate this workflow using Nix and the NixOS integration test framework, where complete integration test environments, consisting of networks of VMs or containers, and all its dependencies are defined from the ground up and executed identically both locally and in CI.

We will outline what it takes to adopt this philosophy in practice, discuss its limitations, and provide concrete guidance for applying it in existing projects.

Related Sessions

Virtual Pass session
Wed, Nov 18 • 9:15 a.m. – 10:15 a.m.
Room F1+F2+F3 - Plenary

45-minute Keynote

Collaboration & Communication Testability Testing Tools

Virtual Pass session
Wed, Nov 18 • 11:45 a.m. – 12:30 p.m.
Room F3 - Track 3: Talks

25-minute Talk

Collaboration & Communication Continuous Integration/Continuous Delivery (CI/CD) Quality Coaching

Virtual Pass session
Tue, Nov 17 • 10:45 a.m. – 11:30 a.m.
Room E1 - Track 4: Talks

25-minute Talk

Continuous Integration/Continuous Delivery (CI/CD) Quality Coaching Testability

Virtual Pass session
Wed, Nov 18 • 1:30 p.m. – 2:15 p.m.
Room D7 - Track 9: Talks

25-minute Talk

Test Automation