Tracer — Sample Audit Evidence Pack
One assessment, three altitudes: the executive summary, the manager repair queue, and the record-level detail all derive from the same evidence model, so any number above can be followed down to the records that produce it — the summary never says more than the detail supports.
Executive summary
An auditor asking for proof today would find 8 requirements without passing test evidence (5 never tested, 3 tested without a recorded pass) and 8 data-quality gaps in the source records.
Manager view
Repair list — the prioritized queue
Ordered by audit exposure: untested requirements by priority first, then requirements whose linked tests have never passed, then unlinked tests, then data-quality repairs. The ordering rule is deterministic and stated so two people reading this list start at the same row.
| # | Kind | Records | Action |
|---|---|---|---|
| 1 | untested_requirement | REQ-007 | Write and run at least one test for REQ-007 (“PDF and image uploads accepted”, priority medium). |
| 2 | untested_requirement | REQ-019 | Write and run at least one test for REQ-019 (“Permission changes written to an audit log”, priority medium). |
| 3 | untested_requirement | REQ-025 | Write and run at least one test for REQ-025 (“Passwords rotated on a 90-day policy”, priority medium). |
| 4 | untested_requirement | REQ-013 | Write and run at least one test for REQ-013 (“Reminder sent for drafts inactive 14 days”, priority low). |
| 5 | untested_requirement | REQ-024 | Write and run at least one test for REQ-024 (“Monthly application volume trend report”, priority low). |
| 6 | no_passing_evidence | REQ-005 | Get a passing run recorded for REQ-005 — its linked tests have no passing result. |
| 7 | no_passing_evidence | REQ-010 | Get a passing run recorded for REQ-010 — its linked tests have no passing result. |
| 8 | no_passing_evidence | REQ-018 | Get a passing run recorded for REQ-018 — its linked tests have no passing result. |
| 9 | unlinked_test | TC-015 | Link TC-015 to the requirement it verifies, or retire it. |
| 10 | unlinked_test | TC-028 | Link TC-028 to the requirement it verifies, or retire it. |
| 11 | unlinked_test | TC-033 | Link TC-033 to the requirement it verifies, or retire it. |
| 12 | unlinked_test | TC-040 | Link TC-040 to the requirement it verifies, or retire it. |
| 13 | data_quality | DEF-008 | Field “severity” is missing. Assign a severity so this defect is counted in exposure by severity. |
| 14 | data_quality | REQ-009, REQ-016 | Circular dependency: REQ-009 depends on REQ-016 depends on REQ-009. Break the cycle; a dependency loop makes sequencing and impact analysis undecidable. |
| 15 | data_quality | REQ-011 | Field “priority” is missing. Set a priority so defect exposure and planning can rank this requirement. |
| 16 | data_quality | REQ-022 | Field “priority” is missing. Set a priority so defect exposure and planning can rank this requirement. |
| 17 | data_quality | TC-012 | Field “last_result” is missing — no execution recorded. Run the test or mark it not-run; a linked test with no result gives no coverage evidence. |
| 18 | data_quality | TC-021 | Broken link: requirement “REQ-099” does not exist in the dataset. Point the test at an existing requirement or remove the stale link. |
| 19 | data_quality | TC-031 | Field “last_result” is missing — no execution recorded. Run the test or mark it not-run; a linked test with no result gives no coverage evidence. |
| 20 | data_quality | TC-040 | Field “last_result” is missing — no execution recorded. Run the test or mark it not-run; a linked test with no result gives no coverage evidence. |
Provenance
| source_system | seed/sample-product.json |
|---|---|
| source_id | REQ-007, REQ-019, REQ-025, REQ-013, REQ-024, REQ-005, REQ-010, REQ-018, TC-015, TC-028, TC-033, TC-040, DEF-008, REQ-009, REQ-016, REQ-011, REQ-022, TC-012, TC-021, TC-031 |
| extraction_logic | scripts/evidence-model.mjs#repair_list |
| query_timestamp | 2026-01-15T00:00:00Z |
Practitioner detail
1. Traceability
17 of 25 requirements (68%) trace to at least one linked test case whose most recent result is a pass. Rule: a requirement counts as traced only on recorded passing evidence — a linked test that failed, or was never run, does not count.
Provenance
| source_system | seed/sample-product.json |
|---|---|
| source_id | REQ-001, REQ-002, REQ-003, REQ-004, REQ-006, REQ-008, REQ-009, REQ-011, REQ-012, REQ-014, REQ-015, REQ-016, REQ-017, REQ-020, REQ-021, REQ-022, REQ-023 |
| extraction_logic | scripts/evidence-model.mjs#traceability |
| query_timestamp | 2026-01-15T00:00:00Z |
Requirements with linked tests but no passing evidence — 3
| Requirement | Title | Area | Linked tests (result) |
|---|---|---|---|
| REQ-005 | Quote recalculates when loan term changes | Rate Quotes | TC-009 (fail) |
| REQ-010 | Reviewer decision recorded with timestamp | Review Workflow | TC-012 (no result) |
| REQ-018 | Session ends after 15 minutes of inactivity | Access Control | TC-026 (fail) |
Provenance
| source_system | seed/sample-product.json |
|---|---|
| source_id | REQ-005, REQ-010, REQ-018 |
| extraction_logic | scripts/evidence-model.mjs#traceability |
| query_timestamp | 2026-01-15T00:00:00Z |
2. Untested requirements
5 of 25 requirements (20%) have no linked test cases at all.
| Requirement | Title | Area | Priority |
|---|---|---|---|
| REQ-007 | PDF and image uploads accepted | Document Upload | medium |
| REQ-013 | Reminder sent for drafts inactive 14 days | Notifications | low |
| REQ-019 | Permission changes written to an audit log | Access Control | medium |
| REQ-024 | Monthly application volume trend report | Reporting | low |
| REQ-025 | Passwords rotated on a 90-day policy | Access Control | medium |
Provenance
| source_system | seed/sample-product.json |
|---|---|
| source_id | REQ-007, REQ-013, REQ-019, REQ-024, REQ-025 |
| extraction_logic | scripts/evidence-model.mjs#traceability |
| query_timestamp | 2026-01-15T00:00:00Z |
3. Unlinked test cases
4 of 40 test cases (10%) are linked to no requirement. Their results cannot support any traceability claim.
| Test case | Title | Last result |
|---|---|---|
| TC-015 | Smoke: portal home page loads | pass |
| TC-028 | Legacy importer accepts v2 file format | fail |
| TC-033 | Load test: 200 concurrent applications | pass |
| TC-040 | Accessibility sweep of application form | no result |
Provenance
| source_system | seed/sample-product.json |
|---|---|
| source_id | TC-015, TC-028, TC-033, TC-040 |
| extraction_logic | scripts/evidence-model.mjs#unlinked_tests |
| query_timestamp | 2026-01-15T00:00:00Z |
4. Open defect exposure
8 of 12 recorded defects are open, 4 at high or critical severity. Grouped by severity; “unspecified” means the severity field is missing on the record (see data-quality findings).
| Severity | Open | Defects and areas |
|---|---|---|
| critical | 1 | DEF-001 — Review Workflow |
| high | 3 | DEF-002 — Rate Quotes DEF-003 — Access Control DEF-004 — Application Intake |
| medium | 3 | DEF-005 — Document Upload DEF-006 — Notifications DEF-007 — Reporting |
| unspecified | 1 | DEF-008 — Document Upload |
Open defect detail
| Defect | Title | Severity | Area | Requirement |
|---|---|---|---|---|
| DEF-001 | Decision record lost when reviewer session expires mid-save | critical | Review Workflow | REQ-010 |
| DEF-002 | Quote does not recalculate when loan term changes | high | Rate Quotes | REQ-005 |
| DEF-003 | Session stays active past the configured timeout | high | Access Control | REQ-018 |
| DEF-004 | Resumed draft drops co-borrower income fields | high | Application Intake | REQ-002 |
| DEF-005 | Oversize upload error shows raw error code to borrower | medium | Document Upload | REQ-008 |
| DEF-006 | Status-change email sent twice for a single event | medium | Notifications | REQ-012 |
| DEF-007 | Pipeline report excludes applications submitted same day | medium | Reporting | REQ-014 |
| DEF-008 | Upload scan result not stored with the file record | unspecified | Document Upload | REQ-022 |
Provenance
| source_system | seed/sample-product.json |
|---|---|
| source_id | DEF-001, DEF-002, DEF-003, DEF-004, DEF-005, DEF-006, DEF-007, DEF-008 |
| extraction_logic | scripts/evidence-model.mjs#defect_exposure |
| query_timestamp | 2026-01-15T00:00:00Z |
5. Data-quality findings
8 findings in the source dataset itself: missing fields, broken links, and dependency cycles. These limit how far the numbers above can be trusted, so they are reported alongside them.
| Location | Finding | Suggestion |
|---|---|---|
| defects/DEF-008 | Field “severity” is missing. | Assign a severity so this defect is counted in exposure by severity. |
| requirements/REQ-009 → REQ-016 → REQ-009 | Circular dependency: REQ-009 depends on REQ-016 depends on REQ-009. | Break the cycle; a dependency loop makes sequencing and impact analysis undecidable. |
| requirements/REQ-011 | Field “priority” is missing. | Set a priority so defect exposure and planning can rank this requirement. |
| requirements/REQ-022 | Field “priority” is missing. | Set a priority so defect exposure and planning can rank this requirement. |
| test_cases/TC-012 | Field “last_result” is missing — no execution recorded. | Run the test or mark it not-run; a linked test with no result gives no coverage evidence. |
| test_cases/TC-021 | Broken link: requirement “REQ-099” does not exist in the dataset. | Point the test at an existing requirement or remove the stale link. |
| test_cases/TC-031 | Field “last_result” is missing — no execution recorded. | Run the test or mark it not-run; a linked test with no result gives no coverage evidence. |
| test_cases/TC-040 | Field “last_result” is missing — no execution recorded. | Run the test or mark it not-run; a linked test with no result gives no coverage evidence. |
Provenance
| source_system | seed/sample-product.json |
|---|---|
| source_id | DEF-008, REQ-009, REQ-016, REQ-011, REQ-022, TC-012, TC-021, TC-031, TC-040 |
| extraction_logic | scripts/evidence-model.mjs#data_quality |
| query_timestamp | 2026-01-15T00:00:00Z |
6. What this pack does not cover
Tracer reports only what its connected evidence supports. This pack does not cover:
- Code and version control — no repository lane is connected; nothing here speaks to code review or branch state.
- CI and build evidence — test results come from the dataset's recorded outcomes, not from a pipeline.
- Live systems — this is a sample generated from a synthetic demonstration dataset, not a customer environment.
- Requirements completeness — the pack measures the requirements it was given; it cannot detect requirements that were never written down.