RumataVerification protocol
How to prove that backends, workflows, AI steps and frontends work, what counts as evidence, and how to test signed-in apps safely.
Step-by-step guidance.
Before you start
- A running development environment for the app under test.
- Tests can create records and trigger integrations. Know what the test will change before running it.
Evidence, not configuration
A saved graph, an enabled binding, a queued deployment or a clean reload is not proof. Prove behavior by exercising the real path and reading the result independently. Report exactly what was observed and what was not verified.
| Claim | Evidence that supports it | Not enough |
|---|---|---|
| Data persists | Create, read from PostgreSQL, restart the service, read through the API. | A schema, an enabled binding, in-memory state. |
| AI ran | A successful run with an agent step and its returned result. | A saved graph or a plausible final answer. |
| An AI tool ran | A tool call in the run, plus a child run ID for workflow tools. | An attached but unused tool. |
| A release is live | Deployment status ready, then the public address requested. | A queued response or a successful build. |
| Per-user privacy | A second user denied; anonymous rejected. | Hiding the UI. |
Choose the verification tool
| Layer | Tool | Use it to |
|---|---|---|
| Backend | `rumata.debugBackendRequest` | Send app headers and JSON through the managed gateway, and inspect non-2xx results. Also read the service’s OpenAPI and run restart checks. |
| Workflow | `rumata.debugWorkflowRequest` | Run a draft and inspect each node’s resolved input and output. |
| Frontend | Preview interactions, screenshots, network evidence, `get_runtime_errors` | Exercise the interface and build for bundle and binding checks. |
| Browser layout | `preview_set_viewport` | Check desktop, about 840 px and mobile (390 × 844), and always restore `previousViewport`. |
Plan one browser journey for behavior and persistence. Use APIs for test-user setup, ownership probes and cleanup. Batch DOM and API reads in `preview_evaluate`, use `preview_click.waitFor` for async outcomes, and reuse returned snapshots. Stop after demonstrated success; health and console checks alone are insufficient.
Keep a running browser-call count in the verification plan, targeting 20 total with headroom for a repair. Reserve the final runtime check, viewport restoration and cleanup before optional polish. Navigate to the app after its implementation and backend are ready. Complete the main user journey once, run `get_runtime_errors`, then check desktop, tablet and mobile layout without repeating sign-in, uploads or ownership tests at each width. Batch related UI checks, inspect one stable screenshot per viewport, restore the viewport and clean up. Reuse this evidence across skills; repeat only checks affected by a change.
When the browser process has a configured call budget, every tool result reports `verificationBudget` with `limit`, `used` and `remaining`. Research, screenshots, read-only checks and failed browser actions count too; navigation does not reset it. Use the remaining count to reserve completion checks. This feedback does not certify success or relax the caller’s limits, and the same feedback is available through native and public MCP.
For a backend, read the running `/openapi.json` before completion. Check the actual create/list/update/delete operations and their request fields. TypeScript interfaces alone do not generate JSON schemas. Fix omitted runtime route schemas, and verify invalid input is rejected; see Runtime validation and OpenAPI.
Label sandbox and live evidence
Test configured sandbox integrations and label the evidence as sandbox. Missing live credentials block live claims, not sandbox implementation. Disclose limitations and obtain concrete approval before a sandbox release. Never replace live AI with rules, mocks or elevated test entitlements.
Authenticated app checks
- Create test users through the real registration route, then verify sign-in and protected behavior in the browser. Use authenticated API calls for additional users and ownership attacks.
- Use unique, least-privilege test users. Track the exact IDs of everything you create.
- Check that protected behavior works when signed in and that anonymous access is rejected.
- Afterwards delete only those users, their sessions and dependent records, clear browser credentials, and verify removal, including after failed checks. Preserve audit history and pre-existing data.