Skip to content

Requirements traceability

This is the canonical evidence ledger. Each stable requirement and threat ID has one row. States are limited to those defined in Requirements ownership, and a row is verified only when exact implementation and verification evidence exist.

Requirements

ID Source State Specification Implementation evidence Verification evidence Notes
REQ-CORE-001 Core demo implemented Setup CLI; deployment framework scripts/setup.py; utility/src/retail_setup/; deploy/; fabric/ tests/scripts/test_setup_bootstrap.py; utility/tests/test_cli_entrypoint.py A live full-demo run proves the complete guided path; separate core/standard profile proof remains under IMP-012.
REQ-CORE-002 Core demo accepted Access control SECURITY.md; generation modules; dim_customers.tmdl — Field inventory, role review, and consumer tests remain open.
REQ-CORE-003 Core demo verified Data contract utility/src/retail_setup/generation/runtime.py; seeded generation modules utility/tests/generation/test_runtime.py; utility/tests/generation/test_engine.py Deterministic generation is covered at runtime and engine layers.
REQ-CORE-004 Core demo implemented Deployment framework; operations runbook Terraform, deploy scripts, setup notebooks, Fabric item sources, and the profile-aware live verifier Local packaging, readiness, deploy-script tests, and a complete live full-demo deployment Required workspace execution is proven; only recent optional stream freshness remains IMP-013.
REQ-CORE-005 Core demo verified Documentation site contracts/retail-demo.json; workspace inventory; guides, design docs, and README entry points tests/docs/test_solution_manifest_docs.py; utility/tests/contracts/test_solution_manifest.py; clean Zensical build Manifest-owned claims and source-derived counts are contract-tested.
REQ-SETUP-001 Setup requirements implemented Setup CLI scripts/setup.ps1; scripts/setup.sh; scripts/setup.py tests/scripts/test_setup_bootstrap.py CI does not yet exercise every wrapper on its native operating system.
REQ-SETUP-002 Setup requirements verified Setup CLI utility/src/retail_setup/cli/main.py; utility/src/retail_setup/config/generation.py utility/tests/test_cli_configure.py; utility/tests/test_generation_config.py Explicit target and deterministic generation inputs are covered.
REQ-SETUP-003 Setup requirements verified Setup CLI notebook injection/render modules; setup notebooks 01-04; stream-events utility/tests/test_cli_render.py; utility/tests/test_notebook_build.py Rendering is atomic and checked against committed notebook sources.
REQ-SETUP-004 Setup requirements implemented Setup CLI scripts/setup.py; utility/src/retail_setup/cli/main.py utility/tests/test_cli_deploy.py; live full-demo deployment and exact-run recovery Native wrapper coverage remains incomplete across every supported operating system.
REQ-DEPLOY-001 Deployment requirements verified Deployment framework Workspace-derived ignored overlays; environment-local tfvars, backend state, Terraform data, bindings, outputs, and journals tests/deploy/test_deploy_config.py; utility/tests/test_cli_configure.py; utility/tests/test_cli_deploy.py Full concurrent publication still requires separate checkouts because item staging is shared.
REQ-DEPLOY-002 Deployment requirements implemented Deployment framework utility/src/retail_setup/cli/main.py; deploy/scripts/ Build, KQL, offline validation, readiness, CLI deploy tests, and a live full-demo deployment The default Azure CLI path is proven live; alternate authentication remains external under IMP-001.
REQ-DEPLOY-003 Deployment requirements accepted Deployment framework Tenant-aware credential factory and clients; Terraform-resolved task-flow targets; configured staged KQL targets test_auth.py; test_apply_kql.py; test_taskflow.py; test_build_artifacts.py; test_cli_deploy.py Live Azure PowerShell and renamed-Eventhouse smoke evidence remains IMP-001.
REQ-DEPLOY-004 Deployment requirements verified Deployment framework; operations runbook Required/optional deploy state machine, atomic deploy/readiness journals, exact-run terminal pipeline gates, bounded deletion and REST pagination CLI journal/readiness tests, pipeline polling tests, workspace polling tests, and live exact-run recovery/readiness Required live readiness is proven; the optional manual stream remains IMP-013.
REQ-DEPLOY-005 Deployment requirements implemented Deployment framework Shared profile resolver; tiered ML pipelines; two-phase Reporting staging/publication; profile-aware configure/render/Terraform/KQL/task-flow orchestration utility/tests/contracts/test_profiles.py; tests/deploy/test_build_artifacts.py; tests/deploy/test_profile_preflight.py; CLI gating tests; live full-demo deployment Full-demo required ML, Reporting, preview preflight, ontology, agents, and task flow are proven live. Separate core/standard profile proof keeps IMP-012 open; IMP-008 is settled.
REQ-GEN-001 Generation requirements verified Data contract utility/src/retail_setup/generation/schemas.py; gold.py utility/tests/generation/test_schema_contract.py; test_engine.py; test_gold.py Base Silver and Gold inventory is contract-tested.
REQ-GEN-002 Generation requirements verified Data contract utility/src/retail_setup/generation/schemas.py utility/tests/generation/test_schema_contract.py The code contract, not README prose, owns physical schemas.
REQ-GEN-003 Generation requirements verified Data contract seeded runtime and generation modules utility/tests/generation/test_runtime.py; representative generation tests Seed and range behavior are deterministic.
REQ-GEN-004 Generation requirements implemented Data contract utility/src/retail_setup/generation/invariants.py Generation module tests Shared business invariants (operating hours, launch eligibility, return timing, and profile controls) are enforced directly by invariants.py.
REQ-STREAM-001 Streaming requirements verified Event contract contracts/retail-demo.json; driver and KQL sources utility/tests/contracts/test_live_data_contract.py; event and truck lifecycle tests The 18-event source-derived matrix passes; only live Fabric staging evidence remains under IMP-005.
REQ-STREAM-002 Streaming requirements implemented Event contract Fail-closed write_to_eventhouse() with persisted stream identity and idempotent Kusto request metadata; Delta debug sink Streaming duplicate-contract and generated-notebook tests A live Eventhouse injected-failure run remains the external verification gate.
REQ-STREAM-003 Streaming requirements verified Event contract; Fabric analytics KQL mappings; source-derived Silver/Gold routes; attribution facts and Direct Lake terminals All-event matrix, marketing-attribution, live-stream, semantic-model, and truck-lifecycle contract tests All repository routes and named exceptions are verified; live staging evidence remains the IMP-005 boundary.
REQ-STREAM-004 Streaming requirements verified Operations runbook Spark checkpoints, Kusto replay tags, key-based Silver merges, staged historical/Gold publication, Delta version rollback, ag._watermarks, and the live freshness adapter Duplicate/replay, checkpoint-tag, freshness, publication, and rollback tests The remaining IMP-013 evidence is a bounded manual stream with recent watermark, ingestion, and checkpoint signals.
REQ-AN-001 Analytics requirements implemented Fabric analytics fabric/kql_database/*.kql; deploy/scripts/apply_kql.py tests/deploy/test_apply_kql.py; live execution of all 135 combined database-script commands Alternate target/authentication smoke coverage remains under IMP-001.
REQ-AN-002 Analytics requirements implemented Fabric analytics Bronze shortcuts and historical/streaming transform notebooks Notebook contract and deployment packaging tests Live shortcut and transform execution is not fully verified.
REQ-AN-003 Analytics requirements verified Fabric analytics; semantic model Current KQL and DAX calculations KPI status, weighting, label-vocabulary, date-key, grain, and technical-field contract tests Status casing, state, grain, weighting, labels, and time-slice semantics are normalized and guarded.
REQ-AN-004 Analytics requirements accepted Fabric analytics querysets, dashboard templates, and rule definitions — First-class dashboard and Activator deployment remains ENH-001.
REQ-MLAI-001 ML and AI requirements implemented ML and AI contracts Required/optional/experimental notebook groups and pipelines; runtime required-output validator; profile publication gate Profile resolver, pipeline, task-flow dependency, build-phase, CLI fail-closed tests, and live full-demo ML pipelines Required-ML/Reporting proof is complete and IMP-008 is settled; remaining profile coverage is tracked by IMP-012.
REQ-MLAI-002 ML and AI requirements implemented ML and AI contracts contracts/retail-demo.json ml_contracts; corrected notebooks 06-14; required TMDL metadata/measures utility/tests/contracts/test_ml_contracts.py; required/optional ML logic tests; semantic/reference tests; live required, optional, and experimental pipelines Contract and live execution are proven; richer lineage and explainability remain ENH-007.
REQ-MLAI-003 ML and AI requirements implemented ML and AI contracts fabric/lakehouse/30-create-ontology.ipynb tests/deploy/test_ontology_notebook.py; live ontology, Data Agent, and task-flow binding checks The full-demo ontology and both agents are live-validated.
REQ-MLAI-004 ML and AI requirements retired ML and AI contracts — — Mandatory agent-governance metadata is outside the default release; persona-specific experiences remain optional under ENH-003.
REQ-BI-001 Power BI requirements implemented Semantic model Direct Lake expressions and model.tmdl tests/scripts/test_reference_integrity.py Live workspace binding remains a deployment gate.
REQ-BI-002 Power BI requirements verified Semantic model TMDL measures, relationships, hierarchies, and properties KPI status, weighting, label-vocabulary, date-key, grain, and technical-field contract tests Status casing, technical-field visibility, grain, label vocabularies, and date-key slicing are guarded.
REQ-BI-003 Power BI requirements implemented Semantic model Six contract-aligned ML tables; hidden lineage/as-of fields; freshness/limitation measures; two-phase publication ML manifest/TMDL agreement tests; semantic/report reference tests; negative CLI publication-gate tests; live gated Reporting publication Core excludes Reporting. The required validation-before-Reporting gate is proven live and IMP-008 is settled.
REQ-BI-004 Power BI requirements accepted Semantic model PBIP report pages and visual metadata — Accessibility, mobile, current-period, and empty-state review remains ENH-006.
REQ-OPS-001 Operations requirements implemented Operations runbook retail-setup verify; injectable REST/Kusto/SQL adapter; profile-aware items, definitions, task flow, KQL, schedules, and exact-run checks test_fabric_runtime.py; test_verify_readiness.py; pipeline/export/task-flow and CLI tests; live full-demo readiness Required live checks pass; the manual stream remains the optional IMP-013 boundary.
REQ-OPS-002 Operations requirements implemented Operations runbook Atomic redacted readiness report over setup_run_log, ag._watermarks, Eventhouse ingestion/tags, pipeline jobs, model lineage, and alerts Freshness, stale-correlation, aggregation, redaction, live-scaffold tests, and live SQL/model evidence Required freshness is proven; recent optional stream evidence remains IMP-013.
REQ-OPS-003 Operations requirements implemented Operations runbook dry-run, target validation, bounded recreate polling, reset notebook, exact-run adoption, and manual fallbacks Target-access, deletion polling, journal recovery, and CLI tests; live exact-run recovery Destructive recreate remains operator-controlled and should continue to be drilled only in disposable workspaces.
REQ-OPS-004 Operations requirements verified Operations runbook .github/workflows/tests.yml; utility/scripts/run_ci_shards.py; fixture-driven pytest markers utility/tests/test_ci_shards.py; tests/scripts/test_workflow_references.py; hosted release-gate run Ubuntu, Windows, four Spark shards, E2E, docs, notebook drift, and repository contracts pass through the aggregate release gate.
REQ-SEC-001 Security requirements implemented Threat model; controls Stable THREAT-* and SEC-* owners Traceability ID and link audit Control evidence remains conservative where verification is incomplete.
REQ-SEC-002 Security requirements retired Access control Synthetic-only generator and demo boundary — Row-level classification and privacy controls for generated demo records are outside the release scope.
REQ-SEC-003 Security requirements accepted Access control .gitignore; identity-based deploy design; tenant-aware credentials; bearer-token request helpers; tenant-free deploy journals Bearer-header, tenant-propagation, journal-redaction, pinned-dependency, and workflow-reference contract tests Live alternate-auth/renamed-target smoke coverage remains IMP-001.
REQ-SEC-004 Security requirements retired Access control Fabric workspace and item permissions — Field-level restrictions, RLS, and mandatory agent-governance metadata are outside the default demo scope.
REQ-DOCS-001 Documentation requirements verified Documentation site docs/; root and component README links python -m zensical build --clean Zensical is the current site generator; website/ is retired.
REQ-DOCS-002 Documentation requirements verified Documentation site; solution manifest contracts/retail-demo.json; canonical requirements, specifications, architecture, security, guides, backlogs, and evidence ledger Global ID/reference/source contracts in utility/tests/contracts/test_solution_manifest.py; docs/source contracts in tests/docs/test_solution_manifest_docs.py Ownership, references, and authoritative-source boundaries are machine-checked.
REQ-DOCS-003 Documentation requirements verified Documentation site zensical.toml; requirements-docs.txt python -m zensical build --clean Current Markdown, navigation, Mermaid, and internal links build with pinned Zensical.
REQ-DOCS-004 Documentation requirements verified Documentation site; workspace inventory Manifest-driven onboarding, deployment/operations guidance, root/component README links, and Zensical navigation tests/docs/test_solution_manifest_docs.py; tests/scripts/test_setup_bootstrap.py; clean python -m zensical build --clean Prerequisites, actual CLI flags, profiles, support, counts, and navigation are locally verified.
REQ-PUBLISH-001 Documentation requirements verified Documentation site .github/workflows/docs.yml; scripts/publish_versioned_docs.py; orphan gh-pages branch Docs run 29283882198; Pages API reports gh-pages root and built; live site redirects to /latest/ Recheck after changing Pages settings, workflow permissions, or branch publication.
REQ-PUBLISH-002 Documentation requirements verified Documentation site scripts/docs_versioning.py; scripts/publish_versioned_docs.py; Zensical mike provider tests/docs/test_publish_versioned_docs.py; live versions.json Published metadata shows Latest and the highest selected patch per stable minor line.

Threats

ID Source State Controls Specification Implementation evidence Verification evidence Notes
THREAT-001 Threat model retired SEC-001, SEC-002 Access control Synthetic-only generator and Fabric workspace permissions — Row-level privacy controls for generated demo records are outside the release scope.
THREAT-002 Threat model accepted SEC-001, SEC-003, SEC-009 Deployment framework Tenant-aware identity clients, bearer request helpers, ignored local config, and redacted deploy journals Auth, bearer-header, tenant-propagation, and journal-redaction tests Live Azure PowerShell and renamed-target smoke evidence remains IMP-001.
THREAT-003 Threat model retired — ML and AI contracts — — Mandatory persona and prohibited-detail controls are outside the default demo release; richer agent experiences remain optional under ENH-003.
THREAT-004 Threat model verified SEC-007 Documentation site Full-SHA actions with no runtime plugin marketplaces; hash-locked Python sets; exact Terraform provider lock; checksummed Miniforge bootstrap Workflow reference and supply-chain contract tests; pip hash validation; Terraform lock validation OS package managers, package registries, and hosted runner images remain documented external trust roots.
THREAT-005 Threat model implemented SEC-008 Operations runbook Fabric monitoring, run logs, watermarks, and bounded redacted readiness reports Readiness redaction, correlation, atomic-write tests, and live full-demo readiness Required live evidence is complete; recent optional streaming evidence remains IMP-013.
THREAT-006 Threat model verified SEC-001, SEC-009 Deployment framework Workspace-derived local config, isolated Terraform backend/data paths, dry-run, recreate, and fail-closed output reuse State-path, placeholder-output, missing-output, wrong-workspace, and destructive-flow tests Full concurrent publication still requires separate checkouts because item staging is shared.
THREAT-007 Threat model verified SEC-008, SEC-011 Operations runbook Fail-closed checkpoints/watermarks, staged publication with rollback, required deploy/readiness failures, and atomic journals Injected deploy, readiness, publication, rollback, cleanup, pagination, replay tests, and live exact-run recovery Required live readiness is proven; the optional manual-stream evidence remains IMP-013.
THREAT-008 Threat model implemented SEC-003, SEC-010 Documentation site Canonical docs/ source, reviewed nav, generated site/ ignored, orphan gh-pages publication Clean Zensical build, successful Docs workflow, gh-pages and live-site inspection Credential-content review remains required for every new public document.