Performance and console results

Besides the screenshot, the Playwright fixture can record page performance and browser console errors at each pixmoat.snapshot() call. pixmoat compares both with a baseline of their own, like it compares pixels. Accessibility checks covers the third kind of finding.

Both need the fixture. The Playwright reporter (beta) collects neither.

A screenshot that’s byte-for-byte identical to its baseline carries no results. To save time, the client doesn’t upload such a screenshot, and its performance, console and accessibility results aren’t sent either. Only new screenshots and screenshots whose image differs from the baseline, even within tolerance, carry results. To send results for every screenshot, set PIXMOAT_SKIP_MANIFEST=true in the Playwright process. This makes runs slower (see Client environment variables).

Collect performance metrics

Performance collection needs two switches:

1. In the Playwright process, set PIXMOAT_PERFORMANCE=true (or 1). This is the only setting that makes the client collect metrics.
2. Turn on Enable performance metrics collection in Settings → Performance. With it off, pixmoat drops the metrics. performance.enabled: true in .pixmoat.yaml also makes pixmoat evaluate them, but the review page and Performance History only show results while the switch in Settings is on.

performance:
  enabled: true
  regression_threshold: 0.15     # a metric 15% worse than its baseline is degraded
  thresholds:
    lcp: { soft_fail: 2500, hard_fail: 4000 }
    cls: { soft_fail: 0.1, hard_fail: 0.25 }

At each snapshot the client records Largest Contentful Paint (LCP), Cumulative Layout Shift (CLS), Interaction to Next Paint (INP), First Contentful Paint (FCP), Time to First Byte (TTFB), the total transfer size, the JavaScript, CSS and image sizes, and the request count. To skip one snapshot, pass performance: false to pixmoat.snapshot().

The client starts observing LCP, CLS and INP the first time a test calls snapshot(), and the observer only sees page loads after that. On a page’s first load, LCP and INP aren’t recorded and CLS is reported as 0; FCP, TTFB and the sizes are recorded. To measure a page’s first load, call pixmoat.snapshot() once on an earlier page in the same test, then navigate to the page you want to measure.

How a result is decided

Each snapshot’s metrics get one result:

Seven metrics are checked. The CSS size, image size and request count are recorded but never change the result. A threshold you don’t set uses its default:

.pixmoat.yaml reference lists the keys.

Performance in the project navigation opens Performance History, which charts each metric across builds.

Capture console errors

Console capture is off until .pixmoat.yaml turns it on. No environment variable controls it, and Enable console capture in Settings → Console doesn’t make the client capture:

console:
  enabled: true
  capture_levels: [error, warning]
  fail_on: [pageerror, requestfailed, error]
  ignore:
    - "Download the React DevTools"
    - "\\[HMR\\]"

The client records:

• console messages at the levels in capture_levels (default error and warning)
• uncaught page errors (pageerror), always
• failed network requests (requestfailed), always

ignore holds regular expressions. A message that matches one is dropped before upload. Use it for framework and browser-extension noise, such as "\\[vite\\]" or "chrome-extension://". Repeats of the same message are counted once, and each snapshot keeps at most 200 different messages. To skip one snapshot, pass console: false to pixmoat.snapshot(); its messages then go with the next snapshot on that page.

The client starts listening at the first snapshot() call on a page. Errors logged before that, including those from the page’s first load, aren’t recorded. To catch load errors, call pixmoat.snapshot() once on an earlier page in the same test, then navigate to the page you want to check. Each snapshot uploads the messages logged since the previous snapshot on that page.

How a result is decided

The client gives each console message a fingerprint from its level and its text, with IDs, long numbers and timestamps removed and each URL cut to its host name, so the same error has the same fingerprint in every run. pixmoat compares the fingerprints with the snapshot’s console baseline:

A finding that’s already in the baseline never counts again, so a project with existing console errors can accept them once and then only hear about new ones. The allow_known key and Allow known findings in Settings don’t change this.

A page with no console messages gets no console baseline. Nothing is uploaded for it, so the first time an error appears on that page, the result is New, not Violations, and it becomes the baseline when the review is complete.

Read the results in a review

When a screenshot has performance results, the review page shows a tab next to Visual labelled with its result, such as Perf: degraded. Press p to open it. The build’s summary shows the counts, and the Overview grid has a Perf degraded filter chip.

The Performance Metrics panel shows each metric’s baseline, current value, change and status.

Console results don’t show in the review page yet. pixmoat records and evaluates them, but the Console tab, the c key, the console counts in the build summary and the Console violations chip don’t appear. The build’s webhook payload includes a console summary.

Findings when the pixels haven’t changed

A screenshot can match its baseline, within tolerance, and still have a degraded metric or a new console error. What that means for the review depends on the gating mode.

The gating that’s enforced is the one in Settings. pixmoat reads Block review on performance regressions in Settings → Performance and Block review on console violations in Settings → Console when it decides whether a build needs review and whether it can finish. A gating value in .pixmoat.yaml is stored with the build but doesn’t change what blocks the review. For performance, the review page follows the file’s value instead, so leave gating out of the file and set the mode in Settings.

With advisory gating, the default, the performance tab shows an Advisory · informational only chip. Nothing needs a decision, and the review can finish without one.

With blocking gating:

• Performance. A Degraded or Failed result keeps a build open for review even when every screenshot matches its baseline. A Degraded result asks for a decision (“Degraded metrics need a decision:”): approve it to accept it, or reject it and fix the page. Finish review doesn’t appear until every degraded result has a decision, and the footer shows Review performance with the count. A Failed result shows Blocked; it can’t be decided, and it makes the build Rejected when you finish. So does a rejected Degraded result.
• Console. Console violations don’t keep a build open. A build whose only findings are console violations finishes as No changes. When the build is open for another reason, finishing fails until every Violations result has a decision, and the review page has no way to make one. Don’t turn on blocking console gating.

The review page only shows Finish review when at least one screenshot needs a visual decision. A build that’s open only because of performance findings can’t be finished from the review page. It stays open until a newer build on the branch replaces it, or a project admin dismisses it (see Superseded builds).

While a build waits only for these findings, the commit status reports no visual changes and the CI gate verdict is PASS. They block the review in pixmoat, not the merge. If you finish the review and the build becomes Rejected because of them, the commit status fails like any rejected build.

When the review is complete, pixmoat updates these baselines on the build’s branch:

• performance and console results that are New, Unchanged, Clean or Warnings become the new baseline
• a Degraded or Violations result becomes the baseline only if someone approved it
• a Failed result never does

A build that finishes as No changes doesn’t update them. So an unreviewed regression keeps being reported in later builds until someone approves it or the page is fixed.

Review visual changes explains how these findings fit into the rest of the review.