Start free

stableFor developers new to pixmoat

Your first build

In this tutorial you connect one Playwright test to pixmoat and take it through the whole loop:

  1. Capture a screenshot in CI and approve it as the first baseline.
  2. Change the page on purpose and watch the merge request pipeline fail.
  3. Review the change in pixmoat and turn the pipeline green.

It takes about 30 minutes. The terms build, snapshot, baseline, capture complete and review complete are explained in Builds and snapshots and Baselines and branches. You don’t need to read those first.

Before you start

You need:

  • A GitLab project for a web app, where you can push to the default branch and add CI/CD variables. A new test repository is the easiest choice, because this tutorial pushes straight to the default branch.
  • Node.js 18 or later, and an app that Playwright can open in CI. The tutorial’s test opens / on BASE_URL, which defaults to http://localhost:3000.

This tutorial uses GitLab CI. The pixmoat steps are the same on other CI providers; only the pipeline file differs.

1. Create an account

  1. Open app.pixmoat.com and select Create an account.
  2. Fill in Full name, Email address, Organization name, Password and Confirm password, then select Create account.
  3. On the Free plan you’ll see Welcome to pixmoat. Select Start exploring.

You’re signed in and on the Projects page. You own the new organization, so you can create projects in it.

2. Create a project

  1. Select Create your first project (or New project if the organization already has projects).
  2. Fill in:
    • Name: your app’s name.
    • Slug: the short ID your CI uses. Note it down; you’ll set it as PIXMOAT_PROJECT.
    • Default branch: your repository’s default branch, usually main. Builds on this branch set the baselines other branches are compared with.
    • CI Mode: choose Blocking.
  3. Select Create project.

You’ll see the project’s Builds page with No builds yet and a setup checklist. That checklist registers the Playwright reporter, which is in beta and can mark screenshots as removed by mistake. This tutorial uses the fixture instead, so skip the checklist.

3. Create a project access token

  1. In the project sidebar, select Settings, then the Tokens tab.
  2. Select Create token, enter GitLab CI as the Token name, and leave Allow review decisions unchecked.
  3. Select Create. You’ll see “Copy this token now. You won’t be able to see it again.” Select Copy.

The token starts with pxg_. If you lose it, revoke it and create a new one.

Now store it in GitLab: open Settings > CI/CD > Variables and add a variable with the key PIXMOAT_TOKEN and the token as its value. Mask it, but don’t protect it: GitLab only passes protected variables to protected branches, and the merge request in step 6 runs on an unprotected one. Never commit the token to the repository.

4. Capture a screenshot

Install the Playwright client in your repository:

npm install --save-dev @playwright/test @s4labs/pixmoat-playwright

Add these three files at the root of your Playwright project.

playwright.config.ts runs global-setup.ts before the tests and global-teardown.ts after them. If your config already starts your app with webServer, keep that part.

// Playwright config for capturing screenshots with the pixmoat fixture.
//
// File location: playwright.config.ts at the root of your Playwright project,
//                with global-setup.ts and global-teardown.ts next to it.
// Install:       npm install --save-dev @playwright/test @s4labs/pixmoat-playwright
//                npx playwright install --with-deps chromium
// Variables:     PIXMOAT_PROJECT, PIXMOAT_TOKEN, PIXMOAT_BRANCH and
//                PIXMOAT_COMMIT (required), PIXMOAT_URL (default
//                https://app.pixmoat.com). In CI, set the branch and commit
//                from your CI variables; see the CI examples.
// Command:       npx playwright test
// Expected:      each pixmoat.snapshot() call uploads a screenshot, and the run
//                ends with "[pixmoat] Build finished: ..." from global-teardown.ts.
//
// Tests capture screenshots with the fixture, not with toHaveScreenshot():
//
//   import { test } from "@s4labs/pixmoat-playwright";
//
//   test("homepage", async ({ page, pixmoat }) => {
//     await page.goto("/");
//     await pixmoat.snapshot(page, "homepage");
//   });
import { defineConfig, devices } from "@playwright/test";

export default defineConfig({
  testDir: "./tests",
  globalSetup: "./global-setup.ts",
  globalTeardown: "./global-teardown.ts",
  use: {
    baseURL: process.env.BASE_URL ?? "http://localhost:3000",
  },
  projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
});

global-setup.ts starts one build for the whole run and writes its ID to pixmoat-build.env for the gate job:

// pixmoat global setup: starts one build for the whole Playwright run.
//
// File location: global-setup.ts next to playwright.config.ts.
// Install:       npm install --save-dev @playwright/test @s4labs/pixmoat-playwright
// Variables:     PIXMOAT_PROJECT, PIXMOAT_TOKEN, PIXMOAT_BRANCH and
//                PIXMOAT_COMMIT (required; startBuild() throws without the
//                branch and commit), PIXMOAT_URL (default
//                https://app.pixmoat.com), and optionally PIXMOAT_CI_URL and
//                PIXMOAT_PR.
// Command:       npx playwright test
// Expected:      pixmoat-build.env contains PIXMOAT_BUILD_ID=<uuid> after the run.
//
// startBuild() stores the build ID in process.env.PIXMOAT_BUILD_ID, which only
// lives inside the Playwright process. Writing it to pixmoat-build.env hands it
// to the CI gate step that runs afterwards. If PIXMOAT_BUILD_ID is already set,
// startBuild() reuses that build instead of creating one.
import { writeFileSync } from "node:fs";
import { startBuild } from "@s4labs/pixmoat-playwright";

export default async function globalSetup(): Promise<void> {
  const buildId = await startBuild();
  writeFileSync("pixmoat-build.env", `PIXMOAT_BUILD_ID=${buildId}\n`);
}

global-teardown.ts finishes the build once every test has run:

// pixmoat global teardown: marks the build complete after every test has run.
//
// File location: global-teardown.ts next to playwright.config.ts.
// Install:       npm install --save-dev @playwright/test @s4labs/pixmoat-playwright
// Variables:     the same as global-setup.ts.
// Command:       npx playwright test
// Expected:      the run ends with a line like
//                [pixmoat] Build finished: 5 snapshots (3 new, 2 unchanged, 0 diff, 0 missing)
//
// finishBuild() never fails the run, even when screenshots are waiting for
// review or were rejected. Gate CI on the build verdict in a separate step
// (see the gate job in docs/examples/gitlab/.gitlab-ci.yml).
import { finishBuild } from "@s4labs/pixmoat-playwright";

export default async function globalTeardown(): Promise<void> {
  await finishBuild();
}

Then add a test that captures the home page, in tests/homepage.spec.ts. It imports test from the pixmoat client, which gives each test a pixmoat fixture:

import { test } from "@s4labs/pixmoat-playwright";

test("homepage", async ({ page, pixmoat }) => {
  await page.goto("/");
  await pixmoat.snapshot(page, "homepage");
});

Finally, add the pipeline. Save this as .gitlab-ci.yml and set PIXMOAT_PROJECT to your project’s slug. Keep the Playwright image tag equal to the @playwright/test version in your package.json.

# pixmoat visual regression in GitLab CI: a capture job and a merge gate.
#
# File location: .gitlab-ci.yml at the repository root. If you already have
#                one, copy the two jobs and the variables into it.
# Install:       npm install --save-dev @playwright/test @s4labs/pixmoat-playwright,
#                plus playwright.config.ts, global-setup.ts and
#                global-teardown.ts from the Playwright example.
# Variables:     PIXMOAT_TOKEN, the project access token from pixmoat (the
#                project's Settings > Tokens tab), as a masked but not
#                protected CI/CD variable (Settings > CI/CD > Variables), so
#                merge request pipelines on unprotected branches get it. Set
#                PIXMOAT_PROJECT below to your project slug.
# Command:       push a branch and open a merge request.
# Expected:      visual-regression uploads the screenshots and logs
#                "[pixmoat] Build finished: ...". visual-review prints
#                "pixmoat verdict: PASS" and passes, or prints REVIEW,
#                REVIEW_NEW or FAIL and fails until the changes are approved in
#                pixmoat and the job is retried.
#
# To block merges, turn on "Pipelines must succeed" under
# Settings > Merge requests. Don't add allow_failure to visual-review: a job
# that may fail leaves the pipeline green and the review is bypassed.

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

variables:
  PIXMOAT_URL: "https://app.pixmoat.com"
  PIXMOAT_PROJECT: "my-project"

stages:
  - test

# Captures and uploads screenshots with the fixture. finishBuild() never fails
# the job on visual changes; visual-review does that.
visual-regression:
  stage: test
  # Keep the image tag equal to the @playwright/test version in package.json.
  image: mcr.microsoft.com/playwright:v1.52.0-noble
  variables:
    PIXMOAT_BRANCH: $CI_COMMIT_REF_NAME
    PIXMOAT_COMMIT: $CI_COMMIT_SHA
    PIXMOAT_CI_URL: $CI_JOB_URL
    PIXMOAT_PR: $CI_MERGE_REQUEST_IID
  script:
    - npm ci
    - npx playwright test
  artifacts:
    when: always
    reports:
      # global-setup.ts writes PIXMOAT_BUILD_ID=<id> here. The path is relative
      # to the directory the job runs `npx playwright test` from.
      dotenv: pixmoat-build.env

# Reads the build verdict and fails unless it is PASS. wait=true waits for the
# server to finish processing (up to 300 s), not for a human review.
visual-review:
  stage: test
  image: alpine:3.20
  needs:
    - job: visual-regression
      artifacts: true              # brings PIXMOAT_BUILD_ID from the dotenv report
  script:
    - apk add --no-cache curl jq
    - |
      verdict=$(curl --fail --silent --show-error --max-time 330 \
        -H "Authorization: Bearer $PIXMOAT_TOKEN" \
        "$PIXMOAT_URL/api/builds/$PIXMOAT_BUILD_ID/agent-summary?wait=true&timeout=300" \
        | jq -r '.verdict')
      echo "pixmoat verdict: $verdict"
      test "$verdict" = "PASS"
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

Commit all of this and push it to your default branch. The branch pipeline runs only the visual-regression job. Its log ends with:

[pixmoat] Build finished: 1 snapshots (1 new, 0 unchanged, 0 diff, 0 missing)

If the job fails before that line:

  • Missing required pixmoat environment variables: PIXMOAT_TOKEN means the job didn’t get the token. Check the CI/CD variable’s key, and that the variable isn’t protected.
  • A 401 or 403 error means the token is wrong or was revoked. Create a new one in step 3.
  • A navigation error from page.goto means Playwright can’t reach your app. Start it with webServer in playwright.config.ts, or set BASE_URL to where it runs.

Take the first baseline in CI rather than on your laptop. Fonts and rendering differ between machines, so a baseline from your laptop would show as changed in every CI build.

5. Approve the first baseline

  1. In pixmoat, open the project’s Builds page. You’ll see a build for your default branch with the status Uploaded.
  2. Open it. The homepage screenshot is tagged New, because there’s no baseline yet.
  3. Check that the screenshot shows the page you expect. Then select Approve, or press a.
  4. Select Finish review. The Finish Review dialog says “Build will be marked as approved”. Select Confirm.

You’re back on the Builds page, and the build shows Approved. The screenshot is now the homepage baseline on your default branch.

6. Change the UI on purpose

Create a branch, make a visible change, and open a merge request:

git switch -c visual-change
# Change something visible on your home page, such as a heading's text or color.
git commit -am "Change the home page heading"
git push -u origin visual-change

Open a merge request for visual-change. Its pipeline runs two jobs:

  • visual-regression captures the page again. Its log ends with [pixmoat] Build finished: 1 snapshots (0 new, 0 unchanged, 1 diff, 0 missing).
  • visual-review prints pixmoat verdict: REVIEW and fails. The screenshot differs from its baseline and nobody has approved it yet.

The pipeline is red. If you turn on Pipelines must succeed under Settings > Merge requests, GitLab won’t let anyone merge until it’s green.

7. Review the change

  1. In pixmoat, open the Builds page and the new build for the visual-change branch.
  2. The homepage screenshot is tagged Changed. The first build on a branch is compared with the default branch’s baseline, which you approved in step 5.
  3. Compare the baseline with the new capture. Press 1 to 5 to switch between the comparison views, for example the highlighted difference.
  4. The change is the one you made, so select Approve, then Finish review and Confirm.

The build shows Approved, and the new screenshot is now the homepage baseline on the visual-change branch. The default branch’s baseline doesn’t change.

If a change isn’t what you intended, select Reject instead and add a comment. After you finish, the build shows Rejected, and the gate keeps failing until you push a fix and a new build is approved.

8. Turn CI green

A job that already failed doesn’t rerun on its own. In GitLab, open the merge request’s pipeline and retry the visual-review job. This time it prints:

pixmoat verdict: PASS

The job passes and the pipeline turns green. You can merge.

The gate reads the approve and reject decisions in the build. Finish the review before you retry it: only a finished review updates the branch’s baseline.

What happens when you merge

Approving on visual-change didn’t change the default branch’s baseline. After you merge, the next build on the default branch still compares with the old homepage image and shows it as Changed. Approve it in that build, or set up merge promotion so approved baselines follow the merge automatically.

Next steps