How Should You Run End-to-End Tests in a CI/CD Pipeline?

## The short answer

Run end-to-end tests in your CI/CD pipeline as a staged, parallelised gate: a small fast smoke suite on every commit or pull request, and the fuller E2E suite at the points where its slower runtime is justified — before merge, before deploy, or on a schedule. The aim is to give developers quick feedback without letting a slow suite block the pipeline or tempt anyone to skip it. E2E tests only add value in CI if they're fast enough to trust and stable enough to believe.

## Why placement in the pipeline matters

E2E tests are slow. If you run your entire suite on every push, developers wait, builds queue, and people start merging without waiting for green. The art is putting the right tests at the right stage so feedback stays fast where it needs to be and thorough where it can afford to be.

A typical staging looks like:

- **On every commit/PR:** unit and integration tests, plus a small E2E smoke suite covering the few most critical journeys.

- **Before merge or before deploy:** the fuller E2E suite.

- **On a schedule (e.g. nightly):** the longest-running, broadest tests, including any against real external services.

## Run a smoke suite on every change

A smoke suite is a handful of E2E tests that confirm the application's most important journeys still work — can a user log in, can they complete the core action. Keep it small enough to run in a few minutes.

This gives the best of both worlds: real end-to-end confidence on every change, without the wait of the full suite. If the smoke suite fails, you've caught a serious regression early.

## Parallelise to keep it fast

The single biggest lever for E2E speed in CI is parallelism. Split your tests across multiple workers or containers so they run concurrently.

To do this safely:

- **Isolate test data** so parallel tests don't collide (each test owns its data).

- **Shard the suite** evenly so no single worker becomes the bottleneck.

- **Run browsers headless** in CI for speed and stability.

Well-parallelised, a suite that takes an hour serially can finish in minutes.

## Make the environment reproducible

E2E tests need a real, running application. In CI, spin up a clean, ephemeral environment for each run rather than testing against a shared, drifting one.

Good practices:

- **Provision the app and its dependencies in containers** so every run starts identical.

- **Seed a known data baseline** before the suite runs.

- **Stub external services** for most tests so third-party downtime doesn't fail your build.

- **Tear everything down** afterwards to avoid leftover state.

Reproducibility is what turns "works on my machine" into a reliable gate.

## Handle failures intelligently

When an E2E test fails in CI, the developer needs to understand why quickly:

- **Capture artefacts on failure** — screenshots, videos, logs, and traces — so failures can be diagnosed without re-running.

- **Report clearly** which journey failed and at which step.

- **Quarantine known flaky tests** out of the blocking path so they don't block unrelated work, and fix them promptly.

- **Use retries sparingly** and only to absorb genuine infrastructure blips, never to paper over real flakiness.

## Don't let E2E block everything

The full E2E suite should not gate every tiny commit. Reserve the heavyweight runs for meaningful checkpoints, and let fast tests carry day-to-day feedback. If your pipeline is constantly red because of slow or flaky E2E tests, developers will route around it — the worst possible outcome.

A healthy pipeline keeps the fast feedback loop genuinely fast and treats the full E2E run as a deliberate, trusted gate before release.

## Monitor the suite itself

Treat your E2E suite as a system to maintain:

- **Track total runtime** and split it back out if it creeps up.

- **Track flake rates** and act on the worst offenders.

- **Track failure-to-fix time** so red builds don't linger.

At neart.ai, building enterprise-grade products means our pipelines have to deliver both speed and confidence — which is why E2E tests are staged, parallelised, and ruthlessly kept stable rather than thrown in wholesale.

## A pipeline checklist

1. Does a small smoke suite run on every change?

2. Is the full E2E suite reserved for sensible checkpoints?

3. Are tests parallelised with isolated data?

4. Is the environment ephemeral and reproducible?

5. Are failures captured with artefacts and flakes quarantined?

## Takeaway

Stage your E2E tests in CI: fast smoke tests everywhere, the full suite at the moments that matter. Parallelise with isolated data, run in clean reproducible environments, capture rich failure artefacts, and keep flakes out of the blocking path. The goal is a pipeline that's fast enough to trust and stable enough to believe.