PowerFill A54 Fix — Greg-Demo Readiness Completion Report
Author: PSSaaS Systems Architect
Date: 2026-04-19
Status: Code complete; sentinel phase-8-superset-ready-a54-fixed ✓; 233/233 tests pass (32 BestEx + 200 PowerFill + 1 Api + 6 skipped); end-to-end Complete-run achieved on PS_DemoData (run 43e8f148-3d1f-4c40-9f76-a43f84efdfb9); pending Collaborator review and PO push.
Sentinel: phase-8-superset-ready-a54-fixed ✓
Companion docs:
- ADR-021 (PowerFill Port Strategy, amended):
adr-021-powerfill-port-strategy(the new §Narrow Bug-Fix Carve-Out) - Phase 8 W1 completion report:
powerfill-phase-8-completion(the upstream this fix builds on) - A54 / A56 / A65 / A66 (resolution arc + new findings):
powerfill-assumptions-log - PowerFill Dashboards reference (post-fix row counts):
powerfill-dashboards
TL;DR
A54 is RESOLVED. End-to-end Complete PowerFill run is now empirically achievable against PS_DemoData via two surgical fixes inside psp_powerfill_pool_guide (009_CreatePoolGuideProcedure.sql), both within the new ADR-021 §Narrow Bug-Fix Carve-Out:
##cte_posting_set_1300PK extension — addedpa_key NUMERIC(1, 0) NOT NULLto the table CREATE + extended PK from(trade_id, loan_id)to(trade_id, loan_id, pa_key); updated INSERT column list and mirrored_1400's author-intendedpa_keyderivation in the SELECT (identical CASE expression onpool_action's first letter).pt13INNER JOIN qualifier extension — addedAND ps1200_13.settlement_date = pt13.settlement_dateto the JOIN clause. The inline subquery already selected(trade_id, settlement_date)and GROUPed by both — but the legacy JOIN qualified ontrade_idalone, producing a fan-out when a trade has multiple distinct settlement_dates across its loans.
Total surgical surface: 30 lines added in 009_CreatePoolGuideProcedure.sql (29 from PK fix + 1 line of JOIN qualifier; comments add ~16 more lines for traceability).
Empirical proof:
| Run | Status | Step 5 (pool_guide) | Step 6 (ue) | Wall-clock |
|---|---|---|---|---|
| Pre-fix runs (5 historical) | Failed | Dup-key (36177868, 3385000026) (2-col PK) | Not reached | ~25-31s to fail |
5859cd70 (post-PK-only-fix) | Failed | Dup-key (36177868, 3385000026, 3) (3-col PK; bug shape revealed) | Not reached | ~25s to fail |
43e8f148 (post-both-fixes) | Complete | 515 rows | Succeeded (12 log events) | 30s end-to-end |
7c9dfe50 (verification) | Complete | 515 rows | Succeeded (12 log events) | 31s end-to-end |
One A54-resolution-surfaced finding (A66): with Step 6 (ue) now running end-to-end, UE's clear-and-rebuild semantics empty the user-facing run-output tables on syn-trade-empty datasets like PS_DemoData. The post-Complete state shows 0 rows in 7 of 8 user-facing dashboards; only the Hub (Dashboard 1) and pfill_powerfill_log retain visible data. This is documented expected behavior (UE's design intent on a no-arbitrage dataset), not a fix bug. Phase 9 closes this when run against a customer DB with real syn-trade opportunities.
The "Bug as Feature" demo narrative is STRONGER than the kickoff anticipated — PSSaaS surfaced two distinct latent legacy bugs (PK shape + JOIN qualifier omission), shipped both surgical fixes within ADR-021's amended carve-out, and now the proc body completes the full pipeline. The Diagnostic-First Rule application revealed the kickoff hypothesis (multi-pa_key Switching) was wrong — the actual bug was settlement-date variance in the upstream — and the resulting fix is more surgical and more clearly-author-intent-aligned than the kickoff predicted.
Sub-phase calendar time: ~1 Architect-session. Consistent with 6a-6e and Phase 7/8 W1 velocity. The empirical Diagnostic-First step added ~30 minutes vs the no-iteration plan; the second surgical fix added ~10 minutes; both well within Reviewable Chunks scope.
What was produced
Modified files
-
src/backend/PowerSeller.SaaS.Modules.PowerFill/Sql/009_CreatePoolGuideProcedure.sql(+30 LOC, +16 comment LOC) — both A54 fixes insidepsp_powerfill_pool_guideproc body. Three sites: (1)##cte_posting_set_1300CREATE — addedpa_key NUMERIC(1, 0) NOT NULLcolumn + extended PK to 3 cols; (2) INSERT column list — addedpa_key; (3) SELECT — added thepa_keyderivation (identical CASE expression to_1400's SELECT) + extendedpt13JOIN qualifier withAND ps1200_13.settlement_date = pt13.settlement_date. All changes preceded by inline-- A54 FIX (PSSaaS, 2026-04-19)comment blocks with NVO line citations and ADR-021 reference. -
src/backend/PowerSeller.SaaS.Modules.PowerFill/PowerFillModule.cs— sentinel bumped fromphase-8-superset-readytophase-8-superset-ready-a54-fixed(one-line change atMapGet("/status", ...)). -
docs-site/docs/adr/adr-021-powerfill-port-strategy.md— Status amended to "Proposed (2026-04-16) — Amended 2026-04-19 with §Narrow Bug-Fix Carve-Out"; new §"Narrow Bug-Fix Carve-Out" section with 4 acceptance criteria + canonical first instance (A54) write-up + "What this is NOT" guardrails + "Forward-only" note. Revision Triggers section gains 2 new bullets (second carve-out instance triggers a sibling ADR; Greg/Tom rejection triggers reversal). -
docs-site/docs/specs/powerfill-assumptions-log.md— A54 marked RESOLVED with full empirical-resolution arc (5-run signature evolution table); A56 marked RESOLVED as A54-side-effect closure; A54 + A56 disposition entries earlier in the file updated to RESOLVED status; A65 added (multi-pa_key Switching + settlement-date variance are two distinct triggers; both fire on PS_DemoData; PS608 customer DB behavior unknown — Phase 9 probe needed); A66 added (UE clear-and-rebuild-empty on syn-trade-empty datasets — surfaced behavior, not a bug; A58's "preservation across BR-9" framing carries a sub-clarification). -
docs-site/docs/specs/powerfill-engine.md— Phase 6 carry-over checklist item marked DONE; Phase 7 PoC validation note appended with A54 fix update (sentinel + Hub dashboard 11-rows + Cash Trade Slotting 0-row reframe per A66); Phased Implementation table Phase 8 row updated with A54 fix sub-entry; BR-9 amended with A66 note about UE's end-to-end-rebuild behavior. -
docs-site/docs/superset/powerfill-dashboards.md— A54/A56 carry-over rendering section completely rewritten with pre-fix vs post-fix row count table; new canonical proof-of-life identified (Hub Dashboard 1 instead of Cash Trade Slotting Dashboard 8).
New files
docs-site/docs/handoffs/powerfill-a54-fix-greg-demo-readiness.md— this completion report.docs-site/docs/devlog/2026-04-19f-a54-fix.md— devlog entry.
Out of scope (deliberately not produced)
- No new ADR — the fix lives inside ADR-021's amended scope per the kickoff's explicit guidance.
- No backwards-compat path for the old 2-col
_1300PK shape — fix is forward-only per ADR-021 §Narrow Bug-Fix Carve-Out's "strictly more permissive" criterion. - No Phase 9 parallel-validation harness — Phase 9 closes the broader "PSSaaS produces same allocations as Desktop App" question; this fix only closes the runtime block and creates the Phase 9 hooks (per A54 + A65 + A66 dispositions).
- No Phase 8 W2 (React UI) — separate session.
- No Tom escalation — PO has decided demo-as-consultation is sufficient for Greg specifically (per ADR-021 §Narrow Bug-Fix Carve-Out clause d). Documentation captures the decision.
- No syn-trade investigation — A66 documents the finding; the deeper "why does PS_DemoData have no syn-trades?" question is Phase 9 / Greg-consultation territory per Path (i) PO direction.
- No staging deploy — the fix lives in
009_*.sqlmounted into the local pssaas-db container which deploys it to PS_DemoData; staging API + AKS rollout are unchanged (the API container itself is unchanged except for the sentinel, which is purely informational).
Decisions made
| # | Decision | Rationale | Where |
|---|---|---|---|
| D-A54-1 | Option A (PK extension only) → revised to A' (PK extension + pt13 JOIN qualifier extension) after Diagnostic-First reveal | Original Option A from the kickoff was based on the multi-pa_key Switching hypothesis. PoC empirically falsified that hypothesis (dup-key shifted to (trade, loan, 3) — same pa_key=3 for all 3 dups, ruling out multi-pa_key). Mid-run probe revealed 3 distinct settlement_dates for trade 36177868. Second surgical fix added; both fixes still inside ADR-021 carve-out scope. | Empirical PoC + this report §Decisions |
| D-A54-2 | ADR-021 amendment is a §Narrow Bug-Fix Carve-Out within the existing ADR, NOT a new sibling ADR | Per kickoff's "New ADR-026 NOT required" directive. Carve-out is precedent-setting but lives in ADR-021's scope. Future second instance triggers a sibling ADR (added to ADR-021 Revision Triggers). | ADR-021 amendment + Revision Triggers |
| D-A54-3 | Sentinel bumped to phase-8-superset-ready-a54-fixed | Distinguishes pre-fix vs post-fix Phase 8 dashboard state for the PO and any future Architect/Collaborator session. Makes the fix observable from /api/powerfill/status without re-deriving from commit log. Doesn't reset Phase 8 progress (W1 deliverable unchanged). PO confirmed at planning checkpoint. | PowerFillModule.cs + this report §"What was produced" |
| D-A54-4 | Self-implemented, no subagent delegation | Small surface (2 surgical SQL edits + ADR amendment + assumption log entries + completion report); high decision density (ADR-021 amendment is precedent-setting). Required Delegation Categories analysis: no category match (templated entities >3 = no; boilerplate tests >5 = no; mechanical find/replace >10 files = no). Documented as Deliberate Non-Delegation. | This report §Required Delegation Categories |
| D-A54-5 | Demo-as-consultation for Greg per ADR-021 §Narrow Bug-Fix Carve-Out clause (d) | Greg is the immediate downstream consumer + the canonical domain authority who would normally consult on this kind of legacy-proc bug. The Greg-demo IS the consultation. Reversibility is preserved: if Greg's review reveals a deeper concern, the fix can be reverted (the PSSaaS-deployed 009_*.sql is the single source-of-truth). | ADR-021 §Narrow Bug-Fix Carve-Out (d) + this report TL;DR |
| D-A54-6 | Path (i) — Demo as-is with refined narrative; A66 documented; do NOT investigate UE syn-trade preconditions in this session | The empty post-UE user-facing tables on PS_DemoData are documented expected behavior per UE's design intent on a no-arbitrage dataset. Investigating UE's preconditions is Phase 9 / Greg-consultation territory; in-scope expansion would risk turning the demo into a debug session (kickoff's explicit warning). PO confirmed at the Andon-cord checkpoint. | This report §Risk register + Path (i) Q to PO |
Migrations enumerated
This fix ships 0 new schema migrations. The fix lives entirely in:
- The existing
psp_powerfill_pool_guideproc body inside009_CreatePoolGuideProcedure.sql(Phase 6c) - The PowerFill-owned table count is unchanged at 23 (Phase 6e baseline preserved)
- No new C# entities, services, endpoints, or tests
- No new SQL deploys to
pfill_*tables; only the proc-body re-deploy via the existingdocker exec sqlcmd ... -i /docker-entrypoint-initdb.d/powerfill/009_CreatePoolGuideProcedure.sqlpattern
Gate findings
Three-layer Primary-Source Verification Gate (with Backlog re-read pass per Phase 7 CR #1)
| ID | Layer | Finding | Disposition |
|---|---|---|---|
| F-A54-1 | NVO-vs-doc | NVO line ranges in the kickoff (9889-9919 for _1300; 10027-10031 for _1400) were approximate. Direct verification: _1300 CREATE is at NVO 9889-9919 (matches); _1400 CREATE is at NVO 9989-10071 (kickoff cited 10027-10031, which is mid-_1400-CREATE — close enough to find but not byte-precise). The structural anchors (_1300 PK=2 cols, _1400 PK=3 cols incl. pa_key NUMERIC(1, 0) NOT NULL) are exact. | (a) Pass on structural anchors. Banking with A57/A59 pattern: kickoff line ranges drift; structural anchors hold. |
| F-A54-2 | NVO-vs-implementation | 009_CreatePoolGuideProcedure.sql lines 1977-2007 = _1300 CREATE; line 2009 = sole INSERT INTO _1300; lines 2089-2121 = _1400 CREATE (already correct 3-col PK + pa_key NUMERIC(1, 0) NOT NULL); lines 2182-2194 = _1400's pa_key derivation. PSSaaS port faithfully transcribes the legacy bug. | (a) Pass. No drift. |
| F-A54-3 | NVO-vs-tenant-DB (planning time) | PS_DemoData pfill_run_history confirms 5 runs (3 Failed at pool_guide, 2 Cancelled at allocation). The dup-key (36177868, 3385000026) is a stable, reproducible empirical anchor across 6c/6d/6e/7. _1300 is a ## temp table — not deployed to the DB schema — so the fix lives entirely in the proc body. | (a) Pass. No DB-side artifacts to migrate. |
| F-A54-4 | Upstream sourceability of pa_key (planning time) | _1300 already pulls pool_action from _1200 (line 2038, ps1200_13.pool_action); _1400's pa_key is derived from pool_action's first letter, not sourced from upstream. _1300 can derive pa_key from pool_action using the identical CASE expression _1400 uses 80 lines later — no upstream JOIN needed. | (a) Pass — clean-source confirmed at planning time (avoiding the kickoff's STOP-and-report trigger). |
| F-A54-5 | Diagnostic-First Rule (post-PoC) | THE LOAD-BEARING FINDING. Post-PK-only-fix run 5859cd70-... failed with dup-key (36177868, 3385000026, 3) — three identical (trade, loan, pa_key=3) rows, NOT differing pa_key values. Mid-run empirical probe of pfill_powerfill_guide for trade 36177868 revealed 4 rows / 3 distinct settlement_dates. The actual fan-out source is the pt13 INNER JOIN qualifying on trade_id only despite the inline subquery selecting (trade_id, settlement_date). Plan-time hypothesis (multi-pa_key Switching) was wrong. | (c) Fix expanded — second surgical change added (pt13 JOIN qualifier extension). Banking the process observation: empirical Diagnostic-First step is mandatory before sizing carve-out fixes; plan-time hypotheses about legacy proc bug shape are unreliable. |
| F-A54-6 | Backlog re-read pass at planning time (per Phase 7 CR #1 + Phase 8 W1 F-8-BR-1 banked observation) | Re-read session-handoff Backlog at planning time. Backlog row #22 explicitly lists "A54 + A56 remain Phase 9 carry-overs (Phase 7 read APIs that depend on Step 6/UE return empty until A54 closed; Phase 8 dashboards correctly render that state)" — confirming this fix DIRECTLY closes those Phase 9 carry-overs and unblocks the empty dashboards. Backlog row #21d notes "Phase 8 W2 (React UI) deferred to next Architect session per Reviewable Chunks" — confirming the kickoff's "No Phase 8 W2" scope-out is correct. No other Backlog rows are relevant to this fix. | (a) Pass. The Backlog re-read pass per Phase 7 CR #1 is now 2-session corroborated (F-8-BR-1 + this F-A54-6) — eligible for canonical promotion at next process-discipline revision. |
| F-A54-7 | Implementation-vs-runtime (post-fix) | Post-fix runs 43e8f148-... and 7c9dfe50-... both achieve Status=Complete end-to-end. pfill_powerfill_log populated with 12 forensic events. pfill_run_history.output_guide_count = 515. Sentinel phase-8-superset-ready-a54-fixed returned by GET /api/powerfill/status. Test suite 233/233 pass (32 BestEx + 200 PowerFill + 1 Api + 6 skipped — Phase 7/8 W1 baseline preserved exactly). | (a) Pass — comprehensive. All Deploy Verification Gate arms (a sentinel, b live API run, c live DB probe) green. |
| F-A54-8 | A66 surfaced behavior | UE clears + rebuilds-empty user-facing tables on PS_DemoData (no syn-trade-eligible loan/trade pairs satisfying UE's @pfill_map predicates). All 7 user-facing run-output tables + pfill_cash_market_map end at 0 rows post-Complete. Only pfill_powerfill_log retains 12 events. | (b) Documented as A66 — not a bug. Documented expected behavior per UE's NVO design intent. The 688-row Cash Trade Slotting (Phase 8 W1's canonical proof-of-life) is now 0 — UE supersedes Step 4's pfill_cash_market_map per A66 sub-finding. New canonical proof-of-life: Hub dashboard with 11 runs, latest Complete. |
Pattern observation: F-A54-5 is the canonical evidence for Diagnostic-First Rule's value in carve-out fix sizing. The plan-time hypothesis was clean (kickoff's narrative + my confirmation) but empirically wrong. Without the mid-run probe of pfill_powerfill_guide, the PK-only fix would have shipped, the PoC would have failed identically (with shifted dup-key), and the second surgical fix would have been added in a panic-iteration loop. Banking for future kickoffs: "verify the fan-out source empirically before sizing the fix" belongs in any carve-out planning gate.
Alternatives-First Gate
The kickoff stated 3 options; the empirical evidence revised the chosen option:
- Original Option A (kickoff): PK extension only — chosen. Rationale: minimal surface; mirrors
_1400's author-intended shape exactly. - Option A' (revised post-Diagnostic-First): PK extension +
pt13JOIN qualifier extension — actually chosen. Same minimal-surgical spirit; both changes mirror legacy author intent independently. - Option B (kickoff): PK extension + pre-INSERT dedup CTE — rejected. Dedup would arbitrarily pick a winner and change legacy semantics.
- Option C (kickoff): Modify upstream so
_1200never emits multi-pa_key for the same (trade, loan) — rejected (massive scope creep).
The Diagnostic-First reveal (F-A54-5) showed Option B's "dedup" framing was directionally wrong even on its own terms — the duplicates were identical rows, not different pool_actions. The actual missing predicate (pt13.settlement_date = ps1200_13.settlement_date) is not dedup; it's a missing JOIN qualifier. The empirical evidence changed the option-space itself, not just the chosen option.
Required Delegation Categories
Self-implemented (Deliberate Non-Delegation):
Deliberate Non-Delegation: not a category match
Task: All work — 2 surgical SQL edits in 009_*.sql; ADR-021 amendment;
4 assumption-log entries (A54 RESOLVED, A56 RESOLVED, A65 NEW, A66 NEW);
spec amendments (BR-9, Phased Implementation, Phase 6 carry-over,
Phase 7 PoC note); dashboards reference doc rewrite; sentinel bump;
PoC verification (5 runs); this completion report; devlog entry;
session-handoff bump.
Reasons:
- Templated entities > 3: NO (single SQL file, single proc)
- Boilerplate tests > 5: NO (zero new tests; existing 233 preserved)
- Mechanical find/replace > 10 files: NO (each doc edit is bespoke)
- Architecturally load-bearing decisions: YES — ADR-021 amendment is
precedent-setting; Diagnostic-First reveal of fix-shape mismatch
requires the same author who proposed Option A to course-correct;
A66 framing for the Greg-demo narrative requires the full A54+A56+A66
arc context that a subagent without prior session history would lack.
- Subagent delegation round-trip cost > author cost: YES (small surface)
One delegation considered + rejected: A54 RESOLUTION write-up + A65 + A66 + devlog could have gone to a fast subagent per the kickoff's mention. Rejected because: (a) Diagnostic-First reveal requires the empirical context I gathered live; (b) the A65/A66 inter-references depend on the chronological resolution arc; (c) the devlog's "what we learned" section needs the same author who learned it. Round-trip cost (~30 min for prompt + verification + revision) > author cost (~20 min direct).
Reviewable Chunks
Workstream-boundary checkpoint: none required (single workstream — A54 fix).
Mid-session checkpoints EXERCISED:
- Plan-presentation checkpoint (start of session) — presented the Three-layer Gate findings + Alternatives-First framing + sentinel decision; PO confirmed Option A + sentinel bump before any code changes
- ANDON #1: Diagnostic-First reveal — post-PK-only-fix PoC failure prompted STOP-and-report; presented the empirical fan-out finding + refined fix shape; PO confirmed Option A' (PK + JOIN qualifier) before second surgical change
- ANDON #2: A66 surfaced finding — post-both-fix Complete-but-empty-tables outcome prompted STOP-and-report; presented the A66 framing + 2 paths forward (demo as-is vs investigate UE preconditions); PO confirmed Path (i) before continuing documentation
Pattern observation: Reviewable Chunks at sub-checkpoint boundaries (not just workstream boundaries) IS the right granularity for empirical-discovery sessions. The 3 ANDON-cord pulls were the load-bearing structure of this session; without them I would have shipped the wrong fix shape OR continued with an in-scope-expansion that the PO didn't authorize.
Deploy Verification Gate
| Arm | Description | Evidence |
|---|---|---|
| (a) Sentinel signal | PowerFillModule.cs MapGet("/status") returns phase-8-superset-ready-a54-fixed | docker exec pssaas-api curl -s http://localhost:8080/api/powerfill/status → {"module":"PowerFill","status":"phase-8-superset-ready-a54-fixed"} ✓ (post-restart). dotnet build clean (0 warnings). dotnet test shows 233 passed / 6 skipped / 0 failed — Phase 7/8 W1 baseline preserved exactly. |
| (b) Live API run | POST /api/powerfill/run → 202 Accepted; background worker progresses through Pending → PreProcessing → Allocating → PostProcessing → Complete. GET /api/powerfill/runs/{run_id} returns full RunResponse with all 6 step results. | Run 43e8f148-3d1f-4c40-9f76-a43f84efdfb9 (and 7c9dfe50-1b7d-471f-832a-d55053d2e7c5 reproduces). 30-31s wall-clock. See §"PoC verification commands and outputs" below. |
| (c) DB probe — proc deploys cleanly | psp_powerfill_pool_guide + psp_pfill_insert4_pool_guide deploy cleanly to PS_DemoData via existing docker exec sqlcmd ... -i /docker-entrypoint-initdb.d/powerfill/009_*.sql pattern. OBJECT_ID + modify_date query confirms post-fix proc body is live. | modify_date = 2026-04-19 20:00:56 (matches deploy timestamp). |
| (c) DB probe — empirical fix verification | Pre-fix vs post-fix SqlException 2627 dup-key tuple shape change confirms PK extension is live. Mid-run probe of pfill_powerfill_guide for trade 36177868 confirms 3 distinct settlement_dates (the actual fan-out source). | Pre-fix dup: (36177868, 3385000026) (2-tuple); post-PK-only-fix dup: (36177868, 3385000026, 3) (3-tuple, same pa_key=3 for all 3 dups); post-both-fix: no dup; run completes. |
| (c) DB probe — 8/8 dashboard SQL queries against PS_DemoData (post-fix) | All 8 dashboard SQL files run cleanly; row counts verified. | See §"Post-fix dashboard row counts" below. |
Counterfactual Retro
Knowing what I know now, what would I do differently?
-
Plan-time would have included the Diagnostic-First mid-run probe in the gate. The kickoff's "STOP and report if pa_key cannot be cleanly sourced from upstream" warning was correctly written but I treated it as a binary gate (can
pa_keybe sourced? Yes → proceed). The actual question should have been "is the fan-out source what the kickoff hypothesizes?" — answerable only by mid-run probing. Banking for future carve-out planning gates: include an empirical fan-out-source verification step BEFORE sizing the fix. The cost is ~10 minutes (write probe SQL + submit run + grep result); the upside is avoiding a wrong-fix iteration. -
The "Alternatives-First Gate" framing missed the actual option space. I framed Options A/B/C as "shape of the carve-out fix" — but the empirical evidence revealed the option space was actually "is the failure mode multi-pa_key OR settlement-date OR something else?" The right Alternatives-First would have been "what data-shape hypotheses explain the empirical dup-key pattern?" with explicit verification of each. Banking: when planning a carve-out, the Alternatives-First options should include the diagnosis hypotheses as well as the fix shape options.
-
The kickoff's pre-emptive STOP-and-report instruction was DEAD-ON ACCURATE even though the specific predicted scenario didn't match. The kickoff said "If post-fix POST /run reveals additional downstream Step 5 / Step 6 failures we didn't anticipate (latent bugs masked by A54 firing first) — document each as A66+..." — this is exactly what A66 is. Banking the kickoff-author's prescience as a process retro item: pre-emptive STOP-and-report instructions in kickoffs are load-bearing even when the predicted scenario shifts.
-
Sentinel bump was the right call. The new sentinel
phase-8-superset-ready-a54-fixedmakes the fix observable from/api/powerfill/statuswithout re-deriving from commit log. Future Architect/Collaborator sessions cancurl /api/powerfill/statusand immediately see the post-fix state. Banking: for runtime-block remediation fixes that don't reset phase progress, suffix-style sentinel bumps (-a54-fixed) preserve phase tracking AND surface the fix. -
The Backlog re-read pass at planning time IS now 2-session corroborated (F-8-BR-1 in Phase 8 W1; F-A54-6 in this session). Eligible for canonical promotion at next process-discipline revision per the existing Phase 7 CR #1 nomination.
-
The "Bug as Feature" demo narrative got STRONGER through the Diagnostic-First detour, not weaker. Pre-detour: 1 surgical fix (PK extension); post-detour: 2 surgical fixes (PK + JOIN qualifier). The detour itself becomes part of the demo narrative — "PSSaaS surfaced 2 latent legacy bugs through empirical PoC iteration; here's the resolution arc" is a more credible story to Greg than "we fixed the bug we predicted in advance". Banking: empirical-iteration arcs make strong demo material for domain-expert audiences who value rigor.
-
A58's "preservation across BR-9" framing was fragile to A66. When A58 was authored, A56 prevented UE from running, so the syn-trades + log + cash_market_map tables were de-facto preserved. With A54 + A56 closed and UE running end-to-end, UE supersedes those tables per its design. Banking: framings that depend on side-effects of other open issues need explicit "if this side-effect goes away, here's how the framing changes" sections. Going back to A58 after A66 is a small textual change but the conceptual lesson is bigger.
-
Sub-phase calendar time: ~1 Architect-session. Consistent with 6a-6e + Phase 7/8 W1 velocity at ~2x complexity (2 fixes + 4 assumption-log entries + ADR amendment + new completion report). Empirical Diagnostic-First detour + ANDON-cord protocol added ~40 minutes total but produced a measurably better fix. Banking: process discipline overhead has positive ROI even on "small" sessions.
PoC verification commands and outputs
Status sentinel (Deploy Verification Gate arm a)
$ docker exec pssaas-api curl -s http://localhost:8080/api/powerfill/status
{"module":"PowerFill","status":"phase-8-superset-ready-a54-fixed"}
Build + tests
$ docker exec pssaas-api dotnet build /app/PowerSeller.SaaS.sln --nologo
Build succeeded. 0 Warning(s). 0 Error(s). Time Elapsed 00:00:05.60
$ docker exec pssaas-api dotnet test /app/PowerSeller.SaaS.sln --no-build --nologo
Passed! - Failed: 0, Passed: 32, Skipped: 0, Total: 32 - PowerSeller.SaaS.Modules.BestEx.Tests
Passed! - Failed: 0, Passed: 1, Skipped: 0, Total: 1 - PowerSeller.SaaS.Api.Tests
Passed! - Failed: 0, Passed: 200, Skipped: 6, Total: 206 - PowerSeller.SaaS.Modules.PowerFill.Tests
# Grand total: 233 passed, 6 skipped, 0 failed (unchanged from Phase 7/8 W1 baseline)
009_*.sql deploy to PS_DemoData (Deploy Verification Gate arm c)
$ docker exec -e PWORD='M0th3rFuck1ng$44$' pssaas-db sh -c \
'/opt/mssql-tools18/bin/sqlcmd -S "hostedps-sql.public.086ea791c2f1.database.windows.net,3342" \
-U "kevin_pssaas_dev" -P "$PWORD" -No -d PS_DemoData \
-i /docker-entrypoint-initdb.d/powerfill/009_CreatePoolGuideProcedure.sql'
PowerFill 009: created psp_pfill_insert4_pool_guide
PowerFill 009: created psp_powerfill_pool_guide
# OBJECT_ID + modify_date confirmation:
$ docker exec ... -Q "SELECT name, modify_date FROM sys.objects WHERE name = 'psp_powerfill_pool_guide'"
name modify_date
psp_powerfill_pool_guide 2026-04-19 20:00:56.200
Empirical-resolution arc (post-fix runs)
$ curl -s -X POST -H "X-Tenant-Id: ps-demodata" -H "Content-Type: application/json" \
"http://pssaas.powerseller.local/api/powerfill/run" -d '{}' | jq .
{
"run_id": "43e8f148-3d1f-4c40-9f76-a43f84efdfb9",
"tenant_id": "ps-demodata",
"status": "Pending",
...
}
# Wait ~32s for terminal status
$ curl -s "http://pssaas.powerseller.local/api/powerfill/runs/43e8f148-3d1f-4c40-9f76-a43f84efdfb9" \
-H "X-Tenant-Id: ps-demodata" | jq '.status, .duration, .summary'
"Complete"
"00:00:30.3097423"
{
"constraint_count": 3,
"candidate_count": 0,
"carry_matched": 0,
"carry_missed": 0,
"candidates_per_constraint": { ... },
"allocated_count": 515, ← Step 4 baseline preserved
"kicked_out_count": 0,
"cblock_count": 0,
"pool_guide_count": 515, ← was 0 pre-fix (Step 5 reached + succeeded)
"syn_trade_base_count": 0, ← UE ran; PS_DemoData has no syn-trades (A66)
"syn_powerfill_guide_all_rank_count": 0,
"syn_powerfill_guide_count": 0,
"pfill_powerfill_log_count": 12, ← was 0 pre-fix (UE log written)
"post_ue_allocated_count": 0, ← UE rebuilt empty per A66
"post_ue_pool_guide_count": 0
}
$ ... | jq '.steps'
[
{ "step_name": "bx_cash_grids", "status": 2, "duration": "00:00:00.0000018" }, ← Skipped per A12
{ "step_name": "bx_settle_and_price", "status": 0, "duration": "00:00:00.1115862" }, ← Succeeded
{ "step_name": "candidate_builder", "status": 0, "duration": "00:00:01.1824978" }, ← Succeeded
{ "step_name": "allocation", "status": 0, "duration": "00:00:24.1886400" }, ← Succeeded (515 alloc)
{ "step_name": "pool_guide", "status": 0, "duration": "00:00:01.2779176" }, ← SUCCEEDED (was Failed pre-fix)
{ "step_name": "ue", "status": 0, "duration": "00:00:03.4309454" } ← SUCCEEDED (never reached pre-fix)
]
Run history before vs after fix (Hub dashboard data)
$ docker exec ... -Q "SELECT TOP 11 CAST(run_id AS VARCHAR(40)) AS run_id, status, started_at_utc, failure_step FROM dbo.pfill_run_history ORDER BY started_at_utc DESC"
run_id status started_at failure_step
43E8F148-3D1F-4C40-9F76-A43F84EFDFB9 Complete 2026-04-19 20:16:45 (NULL) ← post-fix
A8E5BCFD-8402-47C9-A57D-CC78E6FBBD52 Failed 2026-04-19 20:07:50 pool_guide ← pre-second-fix (after PK fix)
8C9E46DD-2889-45D7-AFE3-3CF1656E7938 Failed 2026-04-19 20:06:19 pool_guide ← pre-second-fix
FD5B63CC-B252-4A14-AFDB-BFBC6FD19863 Failed 2026-04-19 20:04:14 pool_guide ← pre-second-fix
5859CD70-F482-4782-8154-EFD0B318D234 Failed 2026-04-19 20:01:15 pool_guide ← post-PK-only-fix (revealed A65 fan-out source)
769245CF-D317-411F-87AA-1D4345FB8489 Failed 2026-04-19 02:49:57 pool_guide ← pre-fix (Phase 7/8 PoC source)
C882F0D3-28C5-4F8E-AFAB-FF09E113B78F Failed 2026-04-18 18:11:59 pool_guide ← historical (Phase 6e PoC source)
52EC1C36-4EA4-4082-80A7-F883188E5A00 Cancelled 2026-04-18 17:40:23 allocation ← historical (cancel demo)
3A52F9EB-E496-4B98-B55E-3087E654D1B2 Cancelled 2026-04-18 17:38:12 allocation ← historical (cancel demo)
909D7F16-AB40-4B30-944C-CA2F8B3BFA7B Failed 2026-04-18 17:36:29 pool_guide ← historical (Phase 6e PoC)
7C9DFE50-1B7D-471F-832A-D55053D2E7C5 Complete 2026-04-19 20:23:10 (NULL) ← post-fix verification
11 rows total; latest is Complete; visible Failed → Complete transition is the canonical demo signal.
Post-fix dashboard row counts (post-A54-fix; sentinel phase-8-superset-ready-a54-fixed)
| # | Dashboard | Pre-fix (Phase 8 W1 launch) | Post-fix | Why |
|---|---|---|---|---|
| 1 | Hub (37_pfill_run_history_overview.sql) | 5 | 11 | Audit accumulates; latest Complete |
| 2 | Allocation Guide (38) | 0 | 0 | UE clears+rebuilds-empty per A66 |
| 3 | Trade Recap (39) | 0 | 0 | A66 |
| 4 | Pool Switching (40) | 0 | 0 | A66 |
| 5 | Pool Candidates (41) | 0 | 0 | A66 |
| 6 | Existing Disposition (42) | 0 | 0 | A63 defensive query unchanged; data-shape PS_DemoData artifact |
| 7 | Pooling Guide (43) | 0 | 0 | A66 |
| 8 | Cash Trade Slotting (44) | 688 | 0 | UE supersedes Step 4's pfill_cash_market_map per A66 sub-finding |
The new canonical proof-of-life is Dashboard 1 (Hub). It shows 11 runs total with the latest Complete (output_guide_count=515), visibly different from the prior 5 Failed runs. The Failed → Complete transition IS the demo narrative.
"Bug as Feature" demo narrative for Greg
(For PO copy-paste / verbal use during the Greg demo.)
Setup: "Greg, before we get into the demo, I want to show you something we found while building PSSaaS's PowerFill port."
Slide 1 — The empirical evidence:
"We've been running the legacy
psp_powerfill_pool_guidebyte-for-byte against the PS_DemoData snapshot. Every single run failed with the same SqlException — duplicate primary key on##cte_posting_set_1300. The duplicate key value was always(36177868, 3385000026). Five Failed runs in a row, all identical."
Slide 2 — The forensic process (the "Feature" part):
"We treated it as a primary-source bug. The legacy author wrote
_1300with a 2-column PK, then wrote_1400immediately after with a 3-column PK that includespa_key. The author clearly knew the right shape one CTE later — the 2-column PK was an oversight. We mirrored_1400's shape into_1300(5 lines added) and re-ran."
Slide 3 — The Diagnostic-First moment:
"It still failed. Same loan, same trade, but now the dup-key was
(36177868, 3385000026, 3)— three identical rows, all withpa_key=3. So the bug WASN'T multi-pa_key Switching like we predicted. We probed the upstreampfill_powerfill_guidemid-run and found that trade had 4 rows with 3 distinct settlement_dates across its loans. The actual bug was a missing JOIN qualifier — thept13inline subquery selected(trade_id, settlement_date)but the JOIN qualified ontrade_idalone. One more line added (AND ps1200_13.settlement_date = pt13.settlement_date)."
Slide 4 — The result:
"Run completed end-to-end in 30 seconds. Step 5 produced 515 rows. Step 6 (UE) ran successfully. Two distinct latent legacy bugs surfaced, both fixed surgically, both within our amended ADR-021 §Narrow Bug-Fix Carve-Out scope. The fixes are forward-only and strictly more permissive — every input the legacy proc accepted, the new proc accepts; plus inputs the legacy proc rejected now succeed. Phase 9 parallel-validation against your real customer data is the canonical correctness gate, but PSSaaS now demonstrates end-to-end Complete on the snapshot we have."
Slide 5 — The open questions for Greg:
"Two questions for you, Greg, that we'd love your input on:
1. UE's post-allocation pass clears and rebuilds the user-facing tables. On PS_DemoData, the rebuild produces 0 rows because the snapshot doesn't seem to have syn-trade-eligible loan/trade pairs satisfying UE's
@pfill_mappredicates. Is PS_DemoData representative of typical customer data, or is it atypically syn-trade-poor?2. We bypassed Step 1 (
bx_cash_grids) on PS_DemoData per a separate assumption (A12) becausebx_price_flooris null on this snapshot. Could that upstream defer be why UE finds no syn-trades downstream?"
Phase 8.5 (post-2026-04-20) load-bearing slot-in slides
The Phase 8.5 kickoff §"Phase 8.5 is the last demo-blocker..." anticipated two new slides post-Phase-8.5-ship that replace the "demo dry-run on local URL" framing + the "click-through to Superset in new tab" framing. Both are now committed at the Phase 8.5 wrap; details below.
Slide 5b — The live operator workflow against the auth-protected staging URL (replaces the prior "demo dry-run on local URL" slide):
"Greg, this isn't a local-machine demo — we're going to log in to the live staging environment via Keycloak SSO right now. Same auth boundary, same Keycloak realm PSX uses for their Principal app + Superset + Docs. PSSaaS joined the ecosystem auth at Phase 8.5; same single-sign-on you'd use across the whole PowerSeller suite once it's all in production. Let me click
https://pssaas.staging.powerseller.com/app/— see how it bounces me to the Keycloak login page first? After I authenticate, I land on the operator UI and the URL stays HTTPS; my session is the OIDC token, my tenant identity comes from a claim on that token, not from a header any browser could spoof."
Slide 5c — Embedded dashboards inside the PSSaaS UI (replaces the prior "click-through to Superset in new tab" slide):
"Now I'm on the Home page, and what you see below the navigation is NOT a screenshot or a
Click here to open Supersetlink — that's an actual Superset dashboard rendered inline inside the PSSaaS app, via the@superset-ui/embedded-sdk. Same Hub Dashboard 13 you'd see in standalone Superset, but rendered as part of the operator workspace. When I click into a specific report — say, Allocation Guide — the per-report dashboard loads inline below the operator-grade row table. The user never leaves the PSSaaS app to see analytics; the analytics ARE the PSSaaS app. The technical lift was non-trivial — server-side guest-token mint, CSRF handshake (the most-missed piece of Superset embedding per industry knowledge), per-dashboard UUID lifecycle managed by a registration script run inside the platform-Superset pod — but the Greg-demo payoff is the demo narrative coherence: one app, one sign-in, one set of dashboards, one operator workspace."
Risk register
| Risk | Mitigation | Status |
|---|---|---|
| Greg/Tom rejects the carve-out fix | Both fixes are revertible — git revert the single commit. The phase-8-superset-ready-a54-fixed sentinel surfaces the fix; reverting drops back to phase-8-superset-ready and the prior Failed PoC behavior. | Acceptable; reversibility is a designed property of ADR-021's amendment |
| A66 surfaces additional latent bugs in UE that this session didn't catch | Diagnostic-First Rule + ANDON-cord protocol scaled to A66 finding without scope creep; if Phase 9 / Greg-consultation surfaces more, document as A67+ in same family | Per kickoff: "1-2 additional findings the demo can absorb; >2 risks turning the demo into a debugging session" — A66 is the 1st, demo headroom remains |
| Staging API rollout-restart Ghost Deploy timing race | Local-only fix; staging API is unchanged (sentinel is informational, not deploy-blocking); kicked out of scope | Not in scope per kickoff |
| MDX rendering trap in this completion report (per Phase 6e/7/8-W1 lesson) | Pre-push docs-build check planned in §Pre-push checklist below | Will be verified before commit |
Pre-push checklist
-
dotnet buildclean (0 warnings, 0 errors) -
dotnet testpasses 233/233 (32 BestEx + 200 PowerFill + 1 Api + 6 skipped) - PS_DemoData proc deploy verified (
OBJECT_ID + modify_date) - PoC end-to-end Complete reproducible (run
43e8f148-...+7c9dfe50-...) - All 8 dashboard SQL queries probe cleanly (no SqlException)
- ADR-021 amendment + assumption log + spec + dashboards + completion report + devlog all written
- Pre-push docs-build check (
docker build -f docs-site/Dockerfile.prod docs-site) — to be run by Architect before commit - Session-handoff bumped — to be done in same commit batch
- Atomic commits prepared (Architect commits; PO pushes per
.cursorrules)
What this enables next
- Phase 8 W2 (React UI) — ready to start as planned per Backlog row #21d / W2 kickoff. The runtime block is closed; W2 can build on a working end-to-end pipeline.
- Phase 9 (Parallel Validation Harness) — can now use the pre-fix vs post-fix dup-key signature evolution as a known-good test fixture. The harness's "data-shape compatibility" pre-flight (per A65) is now well-specified.
- Greg-consultation followup — A66's syn-trade question is the natural next consultation point. The Greg-demo IS the consultation per ADR-021 §Narrow Bug-Fix Carve-Out clause (d); whatever Greg surfaces becomes A67+ findings.
- Tom-consultation (optional) — if Greg surfaces a deeper concern about either of the carve-out fixes, Tom (the maintainer of the Desktop App) is the next escalation point. The empirical evidence + ADR-021 amendment make the case prepared.