Testing settings

Choose how much testing a project gets — how deep test cases go, whether Fabric drafts them at all, which environments runs target, and how often CI results are pulled in.

Not every project needs the same testing. A prototype and a regulated production system both benefit from test cases, but asking for security and end-to-end coverage on a two-week spike wastes effort and credits. Fabric's testing settings let you set that level once per project, and everything downstream follows it.

Two pages hold the configuration, both under Project ▸ Settings:

Testing — the testing policy: depth, required test types, evidence, result sync, and which pull-request review lenses run.
Environments — the deployed targets a Fabric-driven run can point at.

Can't see these pages? Both are part of the Testing feature and appear only when it's enabled for your deployment. If Settings ▸ Testing is missing entirely, the Testing feature is switched off — ask whoever administers your Fabric instance to enable it. This is the first thing to check when a project looks like it has lost its testing configuration.

Decide whether Fabric writes test cases at all

Three switches sit at the top of Settings ▸ Testing, under Test-case generation. They decide whether test cases are generated, when in a feature's life that happens, and what Fabric does when one fails. All three save immediately — unlike the policy below them, which is a draft until you press Save.

Switch	Default	What it does
Generate manual test cases	On	Fabric drafts test cases from a feature's acceptance criteria
Apply TDD approach	Off	Test cases are written before implementation, and the requirements are reviewed against them
Open bugs for failing tests	Off	When your CI reports a linked case as failing, Fabric opens a bug for it automatically

"Open bugs for failing tests" opens at most one bug per case. A case that keeps failing does not accumulate duplicates, and a bug that is already open is not reopened. Leave it off if you would rather triage failures yourself from the Runs segment and promote only the ones worth tracking.

Turning Generate manual test cases off stops the drafting entirely — no cases are produced and no credits are spent on them. Everything else in the Testing tab keeps working: you can still author cases by hand, sync them, and read CI results.

Apply TDD approach changes the order of the feature lifecycle:

Default order (TDD off)

Requirements are generated, the feature is implemented, the feature is reviewed against its requirements, and then test cases are drafted from the finished specification.

Test-first order (TDD on)

Test cases are drafted straight after the requirements — before any implementation. The requirements are then reviewed against the flows those cases describe, which is where test-first earns its keep: writing the cases exposes gaps in the specification while it is still cheap to fix them.

After implementation

With TDD on, the feature review considers both the requirements and the test cases, and the cases are then updated to match the flows that were actually built.

Set the testing depth

Strategy & depth is the single most consequential choice on the page. It decides two separate things: how thoroughly Fabric explores each acceptance criterion, and which kinds of test case it may write at all.

Each option states what it actually causes, and every line on it is enforced when cases are drafted.

Depth	What gets written
Light	Functional / acceptance cases only — the happy path and the key negative case. No integration, end-to-end, security, performance or accessibility cases.
Standard	Functional, integration and end-to-end cases, covering the main negative paths and obvious edge cases. Security, performance and accessibility only where an acceptance criterion explicitly calls for them.
Enterprise	Everything above, plus security cases (auth, tenant isolation, injection, data leaks), accessibility cases (keyboard navigation, labels, contrast), boundary values, concurrency and tenant-isolation cases.

Light really is lighter. The depth is stated to the AI as an exclusion as well as an inclusion — a Light project is told not to reach for end-to-end or security cases on its own initiative, even where they would obviously add value. That is the point: a lighter tier is a deliberate choice about cost and scope, not an oversight for the AI to correct.

A sceptic role overrides the depth. Depth and sceptic roles are separate controls that both decide what gets written. If you set Light depth but leave the Security Reviewer enabled, you still get security cases — an enabled role is read as a deliberate exception to the tier, not a conflict. To make a tier genuinely narrow, turn off the roles you don't want along with it.

If a case turns up outside its tier, it arrives as Proposed. On a Light project, an integration or end-to-end case the AI wrote anyway is created as Proposed rather than Draft — visible and reviewable, but not counted towards your coverage until you accept it.

The same applies to a case that turns up in a dimension the tier defers. On Light and Standard, a security, performance or accessibility case written without being asked for arrives as Proposed even if it sits at an allowed level of the pyramid.

Nothing is thrown away, and nothing joins your suite unasked. Cases a sceptic role produced are exempt: enabling that role is what asked for them.

Depth applies to newly drafted cases. Changing it does not rewrite cases you already have, and it takes effect on the next drafting run — there is nothing to restart.

Required test types

Under the tier selector, Required test types shows the six kinds a suite can be asked for — Functional, Integration, End-to-end, Security, Performance and Accessibility — and every drafted suite gives each selected kind at least one case.

By default the chips follow your tier, and the page says so: pick Light and you see Functional alone; pick Enterprise and Security and Accessibility join the row. Nothing is hidden behind the tier name — what it requires is on screen.

Click any chip to set your own list instead. From then on the tier no longer decides it, and the page says that too, with a Follow the depth tier again link that hands the decision back.

Performance is never automatic. It is selectable but sits in no tier's default, because a performance case is worth writing where an acceptance criterion names one rather than on every feature. Select it deliberately and every suite will include one.

Clearing every chip does not mean "require nothing". An empty list is how a project says follow the tier, so unselecting the last chip snaps the row back to your tier's set rather than leaving it blank. If you want Fabric to stop drafting cases altogether, that is the Generate manual test cases toggle under Generation — one setting, one meaning.

Each drafted case also arrives tagged with the level of the test pyramid it sits at — unit, integration, end-to-end, or manual — so the coverage matrix is populated without anyone classifying cases by hand. Where the AI declines to say, the case shows as Unset and you can set it yourself.

There is a second depth setting, and it is not this one. Settings ▸ AI Assistant has a Testing Strategy depth — Early-stage / Standard / Production-Enterprise — that controls how deep a generated Testing Strategy document and the per-feature testing analysis go. The depth on this page controls test-case drafting.

The two now use near-identical words on purpose: both describe the same idea of a light, a standard and a full-rigour project, so calling one "Average" and the other "Standard" only made them look unrelated when they are not. They remain separate settings for separate artifacts — changing one does not change the other, and each page says which artifact it governs.

Tune the rest of the policy

Everything below the depth selector is a draft until you press Save, and Discard returns you to the last saved values.

Evidence policy

What proof a run must capture before a verdict is recorded: Screenshot required (the default), Optional, or None.

Required test types (sceptic roles)

Adversarial reviewers that add extra cases during planning, each with its own lens — Security Reviewer, UX Skeptic, Performance Critic, Accessibility Auditor, and Edge-Case Hunter. All five are on until you change them. Cases they contribute arrive as Proposed, so nothing enters your suite without review.

A role you enable applies at any depth, and wins over the tier's exclusions. Since all five start enabled, a new project set to Light still gets security, performance and accessibility cases until you turn those roles off — which is the single most common surprise on this page.

Coverage target

The percentage the coverage rings on the Testing tab are measured against. Defaults to 80%.

Devices and browsers

The viewport and browser a Fabric-driven run uses. Defaults are 1920×1080 (with 1366×768 as a second option) and Chromium; Firefox and WebKit are available. The first entry in each list is the one a run uses.

House rules

Free-text rules and implementation notes handed to the runner before it drives a case — anything it should know that isn't in the test case itself.

Required testing sign-offs

How many people must approve a feature before it can be marked Done. Defaults to 0, which switches the requirement off entirely — nothing changes for a project that leaves it alone.

Set it to 2, and a feature cannot move to Done until two different people have signed it off.

Choose which pull-request review lenses run

Settings ▸ Testing ▸ Pull-request review has one switch per lens. Both are on by default, so a project that never opens the page behaves as it always did.

Lens	Cost	Needs
Test coverage	Uses AI, spends credits	A configured AI provider
Circular imports	No AI, no credits — computed	An Atlas analysis of the repository

Turning a lens off stops new runs. Findings it already produced stay readable, because switching a lens off is not the same as deleting somebody's accepted findings.

A lens that is off refuses before spending anything. It does not fetch the pull request, call a model, or read the import graph and then discard the result — the refusal happens first, which is the only version of "off" worth having.

See Pull Requests for what each lens looks for.

Require sign-off before a feature is done

When Required testing sign-offs is above zero, each feature's Testing tab gains a sign-off panel showing who has approved and how many approvals are still needed.

Sign off a feature

On the feature's Testing tab, press Sign off. You can add a short note — "verified on staging against AC 3" — which is stored with your approval.

Change your mind

Press Withdraw my sign-off to remove it. You can only withdraw your own; nobody can remove somebody else's approval on their behalf.

Mark the feature done

Once enough people have signed off, the feature can be moved to Done as usual. Before that, the move is refused with a message naming how many approvals are recorded and how many are needed.

One approval per person. Pressing Sign off twice updates your note — it does not add a second approval. A requirement of two genuinely means two different people, which is the entire point of setting it.

Only "Done" is gated. Moving a feature to To do or Discovery is never blocked — those are ordinary board movements, and blocking them would stop you organising work you haven't finished yet.

The device, browser and environment settings are defaults, not constraints. A run that names its own browser, resolution or environment wins; the stored value only fills the gap.

Choose how often a document refreshes

A document enrolled in auto-refresh carries its own cadence:

Cadence	Refreshes
On every deploy	Whenever a deployment lands, on no schedule at all
Daily	Every day — for a document that tracks something moving that fast
Weekly	Every 7 days
Bi-weekly	Every 14 days (the default)
Monthly	Every 30 days

Each document drifts a few hours off its exact anniversary, on purpose. Fabric spreads documents across a window — up to six hours for Daily, up to a day for the longer cadences — so a batch enrolled on the same afternoon does not all come due in the same minute, forever. The offset is derived from the document itself, so it is stable: a Daily document refreshes at roughly the same time each day, not at a random hour.

On every deploy

On every deploy replaces the schedule entirely. Fabric treats a successful CI run on the branch it watches for test results as a deployment.

Set the cadence

On the document, open the auto-refresh settings and choose On every deploy.

Ship

When Fabric next pulls CI results and sees a successful run on that branch, the document is marked for refresh.

Review the update

The refresh behaves exactly as a scheduled one: it drafts an update for you to accept, unless you have turned on Apply automatically.

The refresh starts on the next sweep, not the instant you deploy. A deploy marks the document as due and Fabric's hourly pass does the work. That keeps every safeguard a scheduled refresh has — it will not touch a document somebody is editing, and it will not run under someone who has lost access to the project.

A failed build is not a deploy. Only a successful run counts, and a status Fabric doesn't recognise is treated as "not a deploy" rather than guessed at — a build going red should never spend your credits rewriting documents. If your pipeline reports success under an unusual status, the refresh simply waits for one it recognises.

Control how often CI results arrive

Automatic result sync decides whether Fabric pulls test results from your connected CI on its own, and how often — every 15 minutes (default), 30 minutes, 1 hour, or 4 hours.

Turning it off does not remove the manual path: Sync now in the Testing tab always works. The interval is a floor, not a rival schedule — Fabric will not check more often than you allow.

Intervals are offered as a fixed set rather than a free-text schedule on purpose. A cron box invites * * * * *, and a project quietly overwhelming its own CI provider's API is a worse outcome than a slightly staler result.