GrailTestAI — Observant AI Test Automation

Pipeline (drag any box to rearrange — it sticks across reloads)

← Overview

Decision audit (every orchestrator decision, the rule it applied and the measurements it applied it to)

autoscroll

Stage

Generated files waiting

the real files on disk, exactly as the agent wrote them — artifact/ while it's still working, best/ once an iteration is kept

Not started yet.

Select a file on the left once generation starts.

Live test execution waiting

0 tests observed

Start the execution stage to watch the real Playwright suite run here.

Agent reasoning

autoscroll

Agent reasoning autoscroll

Recovered contract (what discovery read, and the contract every later stage works from)

Select a stage

Loss / best-loss curve (shaded area = AUC · click a point for detail)

Scoring benchmark

Component evolution (why each score is what it is, across every iteration — click a chart to drill in)

Iteration ledger (latest first — click a row for detail)

Feedback loop

No feedback raised in this run.

Human review — classifier candidates as

The latest classified candidates for this section. The snapshot review above follows the selected iteration. Accept keeps a candidate; Reject removes it on the next refinement. Decisions apply to unchanged candidate content and survive a restart.

AI assurance — evidence pack

no pack yet

Derived from this run's own artifacts — the clean-profile verdicts, the seeded-weakness drill, the findings filed — joined to the target's obligations and the framework controls each cites. Nothing here is written by a model.

Waiting for stage 6 to produce an assessment report…

Requirement traceability

Live updates

Waiting for pipeline artifacts…

Hover for details; click to select. Drill up isolates one root-to-item lineage, without sibling branches. Drill down isolates every branch below the item. Back restores the previous view. Double-click toggles children. Wheel zooms; full titles are in the detail panel.

View Layout Keep map in frame Find item Select an item to drill Flight speed

Arrows show source → destination. Solid links are declared; dashed links are inferred. Hover: white links come in, gold links go out. Test results: ✓ passed, ✕ failed, ? unknown. Moving dots show direction, not live execution.

3D flight

WASD

W/S fly · A/D strafe · Q/E down/up · Shift boost

Hold left mouse to look · wheel sets speed (1.0×)

Hold left mouse + WASD to fly · Shift-drag pans

Code coverage (measured on the system under test, never read by agents)

Not collected yet — runs automatically after stage 6.

Source

Mutation testing (requirement-scoped counterfactuals that must be answered by observable tests)

Not collected yet — runs automatically after stage 6.

Round ledger

Survivors raise hidden objectives; the next round verifies whether test changes kill them.

Detection by file

Detected / valid mutants. Invalid and ignored mutants do not dilute the score.

Every mutation, one row each

Human audit view — exact implementation details are never included in agent feedback.

Channel parity fuzzing (same instruction, both channels, no example-based oracle needed — a disagreement is a defect by definition)

Not collected yet — runs automatically after stage 6, only if the target declares parity channels.

Blind-spot self-critique (stage 5's own predictions of where its suite might miss a defect, checked against real mutation survivors)

Not collected yet — checked automatically once mutation testing finishes.