// SHILOH TEST · QA INTELLIGENCE PLATFORM

Ship a build.
Get back evidence.

A URL, an APK, an FRS document, or just screenshots — that's all Pulse needs. AI drafts the test cases, runs them, and reports back. You keep the judgment calls on what ships.

Web · Android · iOS soon AI-Drafted, Human-Approved Zero Setup Gate
// THE QA PAIN MAP

Every feature exists because manual QA bleeds time somewhere.

This is the map. Each pain point has a specific answer, built into the product — not bolted on.

01

Requirements arrive in any form — FRS docs, images, or "here's the app, figure it out"

Generation tab

Multi-modal AI generation with a live streaming feed. No format is off-limits.

02

New features land and test cases lag behind

Generation tab

Incremental generation, a coverage Q&A bot with one-click gap-fill, and AI editing.

03

Organising smoke, sanity, and regression packs

Suites / Flows / Execution

Suites as packs, Flows chaining suites, Rounds per manual cycle.

04

Execution is slow, manual drudgery

Execution tab + Agent

One-click manual recording with evidence; one-click automated runs with a live view.

05

Reporting eats a QA lead's entire week

Runs / Reports

Auto pass/fail, AI root cause per failure, executive reports, PDF/CSV export, email alerts.

06

"Can we actually trust this coverage?"

Coverage tab

Traceability to source, case-type mix, honest gap surfacing, a full evidence trail.

// THE LIFECYCLE

Ship it to us. We test it. We ship it back.

Every stage feeds the next without a manual translation step. The FRS is the input to generation. The generated case is the executable spec. The failure is the defect draft. The results are the report.

Generation Execution — Automated Reporting
01

Intake

A URL, an APK upload, an FRS document, or screenshots — however the build already exists.

02

AI Generation

Features → sub-features → typed test cases, identified and watched live as they're discovered.

03

Execution

Manual rounds with one-click evidence, or automated runs via the Pulse Agent with live screenshots.

04

AI Analysis

Every failure classified — real bug, flaky, or environment — and linked to the case it breaks.

05

Reports Out

Executive summary, PDF, CSV, email — generated automatically, not written by hand.

06

Ship Back

With coverage proof, evidence, and a prioritised risk list — ready for the sign-off call.

See exactly how each stage works. Generic examples, illustrations, and before/after comparisons for all six stages.
Explore the full lifecycle →
// GENERATION

Feed it anything. Get back a full test hierarchy.

FRS documents, screenshots, a plain-English description, or just a URL — Generation accepts all of it, and produces a Feature → Sub-feature → Test case hierarchy, streamed live as Claude reads through your material. No prerequisite steps.

  • FRS documents (PDF / .docx) parsed and grounded automatically
  • Screenshots alone are enough — generate multi-modally, no text required
  • Every test case classified as Positive, Negative, or Edge at generation time
  • Traceable via a source reference back to the exact requirement it came from
  • Real runnable code from day one — Playwright for web, WebdriverIO/Appium for Android
GENERATING Source: Loyalty_Points_FRS.pdf
Feature: Loyalty Points
Sub-feature: Redeem points
TC-101 · redeem_points_at_checkoutPOSITIVE
TC-102 · redeem_more_than_balanceNEGATIVE
TC-103 · redeem_at_exact_balanceEDGE
Feature: Tier Upgrades
Sub-feature: Upgrade notification
TC-104 · tier_upgrade_email_sentPOSITIVE
COVERAGE BOT Suite: Loyalty Points
"Do we have any test cases on the OTP flow?"
No coverage found on OTP verification. Proposed: TC-201 · verify_otp_on_redemption
// COVERAGE BOT & AI EDITING

Ask about your test cases. Fill the gap in two clicks.

"Ask about your test cases" answers questions like this strictly from your real cases, citing the exact TC-00N cases that support the answer. When coverage is missing, it proposes a ready-to-use test case with a one-click Add button.

  • Grounded answers only — the bot never invents coverage that isn't there
  • Any test case or whole sub-feature can be edited with a natural-language instruction
  • Edits are previewed first, applied only on confirmation
  • The Coverage dashboard closes the loop — unclassified cases, never-executed cases, missing negative/edge coverage
// SUITES, FLOWS & EXECUTION

Smoke tests, regression packs, manual cycles — however your team already thinks about testing.

The concepts map directly onto constructs your team already knows.

TESTING CONCEPTSmoke / sanity pack
Suite

Name suites as packs — "Smoke — checkout", "Sanity — auth" — each one-click runnable.

TESTING CONCEPTRegression pack
Flow (pipeline)

Chain suites on a canvas — session (cookies/login) carries across stages, so later suites start authenticated.

TESTING CONCEPTTest cycle / round
Execution round (R1, R2…)

Manual execution scoped per round per platform, with per-round reset and reporting.

TESTING CONCEPTCross-platform pass
Platform tabs

Suites tagged Website / Android / iOS — the Execution dashboard groups by platform.

Manual when you need control. Automated when you need speed.

The Execution dashboard is built for speed — every case gets one-click Pass / Fail / Blocked, with an expandable panel only when you want notes and evidence. The Run Suite button dispatches automated specs to a Pulse Agent: a single standalone binary, no Node install needed.

  • Manual: filter by status, case type, epic, or free-text search
  • Automated: live results and live screenshots stream back per test
  • Real-world blockers handled — a waitForOtp helper polls for codes via Twilio or Mac Messages
  • Quick Launch — open any URL on an agent machine for an instant eyeball check
LIVE Suite: Checkout Regression Agent: qa-agent-04
44 PASSED ✓ 2 FAILED ✗ 1 ANALYSING ◌
✓ apply_promo_code0.9s
✓ guest_checkout_flow1.4s
✗ saved_card_autofillFAILED
◈ AI Root Cause

Saved card autofill breaks after the payment-form refactor merged this morning

Confidence: 91%Real Bug
// REPORTING & COVERAGE

Reporting is generated, not written.

Confidence comes from a system that can't flatter itself.

Per automated run

Pass/fail/skip counts and durations, plus AI root-cause analysis on every failure: probable cause, contributing factors, suggested fix, confidence score, and a classification — real bug, flaky, environment, or bad test data.

Executive report per run

Plain-English summary, key risks, recommendations, and trend versus the prior run — exportable to PDF for the people who don't open the dashboard.

Per manual round

CSV export and a formatted PDF execution report — project, platform, round, per-case status, type, and notes.

Coverage dashboard

Case-type mix is visible — a suite that only tests happy paths is immediately obvious. Gaps are surfaced, not hidden: 88% never-run shows as 88% never-run.

Evidence trail

Manual executions attach screenshots and recordings. Automated failures capture screenshots. Every defect carries both.

Alerts

Email notification on run completion, so nobody has to poll a screen waiting for results.

How the AI stays honest: the coverage bot answers only from real cases and cites their IDs. Coverage insights are generated from deterministically computed counts — the AI narrates the numbers, it never produces them. Defect analysis links to the actual failing result, with its stack trace and screenshot.
09:41
Monday · Locked
Shiloh Pulse now
Checkout Regression — 2 failures
AI root cause ready: saved_card_autofill broke after this morning's deploy.
Shiloh Pulse 2m
Onboarding Smoke — run complete
18 of 19 passed. Tap to view the full report.
Shiloh Pulse 1h
Weekly coverage report
Your Monday digest is ready — 6 suites, 214 test cases.
iOS Android
// THE APP

Pulse doesn't wait for you to check.

Every run completion, every failure, every report — pushed straight to your phone the moment it lands. You don't have to babysit a dashboard to know what's happening.

  • Real-time push notifications on iOS and Android
  • AI root cause summarised right in the notification — no dashboard required
  • Daily and weekly digest options, so urgent and routine stay separate
  • Notification rules per project, so you only hear about what matters
// EVERY SCREEN

A native app, not a shrunk-down dashboard.

Eleven screens, one continuous story: summary, live runs, failure evidence, AI root cause, defect triage, and executive reports — each built for a phone from the ground up, not squeezed from a desktop layout.

  • Summary — every project's pulse, with the single most urgent thing surfaced first
  • Runs & Run detail — live status on every suite, with a Stop control while it's running
  • Failure detail & AI root cause — raw error, screenshot, and a fix written for a non-engineer
  • Projects & Project pulse — per-project health, suites, and machine status
  • Defects & Defect detail — triage with repro steps and evidence, from a phone
  • Reports & Change detail — executive summaries, and every AI edit shown as a before/after diff
Summary
4 projects · Mon 17 Aug
⚠ Critical defect on Kalahari Bank

DEF-241 — card payment fails silently on 3DS challenge. Raised 2 days ago, still open.

Pulse
87%
+12 pts vs prior 14 days
87%
142 runs · last 14 days
Runs
6
in flight today
Machines
3/4
agents online
Defects
7
open, none urgent
Runs
Every project · newest first
AllActiveFailedPassed
✳
Checkout Regression
Northwind Retail · now
18 · 1 · 0
✕
Main Suite
Kalahari Bank · 12m ago
41 · 5 · 2
✓
Smoke — iOS
Northwind Retail · 48m ago
24 · 0 · 1
◌
Nightly Full
Aurora Health · 1h ago
0 · 0 · 0
▢
Payments API
Kalahari Bank · 3h ago
9 · 0 · 14
Checkout Regression
Northwind Retail · run 9182
✳ RUNNING ⏱ 02:14
18
passed
1
failed
0
skipped
▢ Stop run
LIVE · 19 OF 26
✓
cart › adds item
1.2s
✕
checkout › applies promo…
15.0s
✳
checkout › pays with card
Failure detail
run 9181 · test 3 of 48
✕ checkout › applies promo code to order total
>_ Raw error
TimeoutError: locator.click: Timeout 15000ms exceeded.
waiting for getByRole('button', { name: 'Apply' })
▤ Screenshot at failure
AI root cause
checkout › applies promo code
◈ AI Root Cause
Category

UI regression

Probable cause

The Apply button now renders inside a lazy-loaded panel that mounts ~2s after the cart, so the click fires before the control exists.

Suggested fix

Wait for the promo panel's loaded state before clicking, or restore the server-rendered Apply button.

Projects
4 you belong to
92
Northwind Retail ★
8 suites · online
›
71
Kalahari Bank
12 suites · online
›
88
Aurora Health
5 suites · offline
›
★ Active project

Northwind Retail is set as active — Runs, Defects and Coverage filter to it until you change this.

Northwind Retail
Project pulse · last 14 days
Project pulse
92%
↗ +4 pts vs prior 14 days
92%
across 38 runs
Suites
8
2 scheduled nightly
Machine
1/1
online · last seen 2m
Defects
2
open on this project
Defects
All projects
OpenCriticalHighResolved
DEF-241Critical
Card payment fails silently on 3DS challenge
Kalahari Bank · Open · open 2d
DEF-238High
Cart total ignores promo code after refresh
Northwind Retail · In triage · open 4d
DEF-233Medium
Appointment picker skips 29 Feb
Aurora Health · Open · open 9d
DEF-241
Kalahari Bank · critical
Card payment fails silently on 3DS challenge
CriticalOpen · 2d
☣ Steps to reproduce
1Add any item to the basket and continue to payment
2Enter a card enrolled in 3D Secure
3Complete the bank challenge screen
4Observe: spinner ends, no confirmation, no error
📎 Evidence · 2
Reports
Executive summaries
Latest report
Northwind Retail — Release 4.2 readiness
Generated 16 Aug · 9 pages
92%
Checkout is release-ready. Two medium defects remain open in Search, both with workarounds documented.
▤ Read
Earlier
Release 4.1 readiness
02 Aug
88%
Q2 quality review
14 Jul
81%
Change detail
Thabo M. · 22m ago
⑂ Checkout › applies promo code
Field changed: steps · expected result
Before
1Open the cart
2Enter promo code SAVE10
3Tap Apply
After
1Open the cart
2Wait for the promo panel to finish loading
3Enter promo code SAVE10
- await page.getByRole('button').click()
+ await promoPanel.waitFor(); ...
Summary
// DEVOPS & AUTOMATION

Anything that can curl can trigger a run.

Runs are dispatched via plain REST — POST /api/runs with a suiteId — including from a CI job. The agent is a portable standalone binary that connects outbound over websockets, so there are no inbound firewall rules to negotiate.

  • Install the agent on a build machine, a QA laptop, or a VM
  • Flows chain suites into regression pipelines, also dispatchable via API
  • Agent state reconciles on API restart — a dead agent can never silently absorb a dispatched run
APIPOST /api/runs
curl -X POST https://api.pulse.shiloh/runs \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"suiteId":"checkout-regression"}'
// START HERE

Ship your first build.

Point Pulse at a URL, upload an APK, or drop in an FRS document. Watch the test hierarchy build itself.

Get Started →