PowerFill Assumptions Log
Every interpretation made during PowerFill reverse-engineering, with source citation. This document exists because Tom and Greg are not available to consult before the port, so we proceed with explicit working assumptions and flag them for future critique.
When Tom or Greg become available, this document is the starting point for their review. Each assumption is numbered and citation-anchored so critique can target specific claims rather than the whole spec.
Status: Working document — updates as interpretation improves Author: AI Architect (reverse-engineered from legacy code) Reviewers pending: Tom (senior dev, PowerBuilder veteran), Greg (hedging/risk domain)
A1: Multi-Pass Allocation — Empirical Structure (REVISED 2026-04-18 by Phase 6b NVO trace)
Original claim (2026-04-16, Phase 0): CONTRADICTED BY EMPIRICAL TRACE.psp_powerfill_conset executes allocation in four discrete passes: exact fit, best fit, fill remaining, and orphan handling.
Revised claim (2026-04-18, Phase 6b): psp_powerfill_conset executes allocation in 3 stages per constraint, inside an outer per-constraint priority loop:
Outer: WHILE @xc <= @xcPop (constraint priority loop, NVO 571)
Per-constraint:
Setup (NVO ~1208-1908) — populate per-constraint candidate slice;
carry computation; initial guide writes;
lockdown tagging
Stage 1 (NVO ~1909-3967) — Core Cycle 1: 12 internal CHECK POINT
diagnostic markers; bubble pass; Single-X
inner loop (WHILE @singleX > 0); reranking
Stage 2 (NVO ~3968-5521) — Reranking + marginal-cost substitution,
capped at 20 iterations OR convergence
Orphan Sweep (NVO ~5524-5810) — Final fallback for unallocated loans;
uses @j counter as infinite-loop guard
Pass labels (Phase 0's interpretation): "exact fit / best fit / fill remaining / orphan handling" do NOT appear in the NVO anywhere. They were Phase 0 invention. The actual structure has named diagnostic markers THIS ENDS STAGE 1 OF CORE CYCLE 1 (NVO 3967), BEGINS STAGE 2 OF CORE CYCLE 1 (NVO 3968), END OF STAGE 2..CORE CYCLE 1 (NVO 5521), and SWEEP BACK THROUGH ORPHANS (NVO 5524) — these are the canonical labels.
Source: Direct empirical NVO trace by Phase 6b Architect, 2026-04-18. NVO 50-5886 (the entire of_get_psp_powerfill_conset_syntax function body) was read; structural anchors verified by Grep on comment markers + control-flow keywords (WHILE, BEGIN Loop, END OF STAGE); Collaborator independently verified all 4 NVO citations (function boundary; 3-stage structure with literal NVO comment markers; cblock DELETE+INSERT lifecycle; @xc/@xcPop loop variables).
Confidence: High — the structural map is empirical NVO fact, not interpretation. The semantics inside Stage 1 (what each CHECK POINT does), Stage 2 (the marginal-cost substitution algorithm), and the Orphan Sweep (the @j-guarded loop) are still interpretation-heavy and best validated via Phase 9 parallel-validation against the Desktop App.
Disposition for Q4 (multi-pass semantics): RESOLVED by Phase 6b empirical trace per Architect Default Option A (PO-confirmed). The conset proc body deploys verbatim per ADR-021; A1's structural revision is the canonical interpretation going forward. Phase 9 parallel-validation is the authoritative correctness gate.
Implications:
- The Phase 6 sub-phase breakdown's reference to "four discrete passes" should be read as "the multi-pass allocation engine with 3 stages per constraint inside an outer per-constraint loop." A spec amendment is NOT needed (the spec doesn't enumerate pass count); the breakdown's text is a planning artifact, not a contract.
- A1.0 (placeholder for the empirical trace) is RESOLVED in this update — the placeholder section below now points back to this revised A1 entry.
- The Phase 9 parallel-validation harness should compare allocations per-constraint and per-stage (not "per-pass"), since the legacy proc emits no pass labels but does emit stage markers.
Needs confirmation from Tom/Greg: Optional. The structure is empirical; the per-stage semantics (Stage 2's convergence criterion, Orphan Sweep's @j cap value) are clear from the NVO but Tom/Greg could confirm intent. Not blocking any sub-phase.
A2: Carry Cost is a Scoring Modifier, Not a Filter
Claim: Carry cost from pfill_carry_cost adjusts the loan-trade score but never eliminates a candidate from consideration.
Source: pfill_carry_cost is referenced in candidate-scoring queries (e.g., SUM(loan_amount*prx_and_carry)/100) but not in WHERE clauses that would filter candidates. This suggests modifier semantics.
Confidence: High. Separate columns like New_Value_PriceOnly and New_Value_PriceandCarry in powerfill_switching_thumbnail.srd reinforce that carry cost is a scoring dimension, not an eligibility rule.
If wrong: We might incorrectly filter loans that the Desktop App would have kept.
Needs confirmation from Tom/Greg: Is there any scenario where carry cost disqualifies a loan?
A3: pool_action = 'Swapped In' is Internal State
Claim: The value Swapped In appears in pfill_powerfill_guide and pfill_pool_guide to represent the destination side of a Switch operation. It's an internal state not shown in top-level reports except the Switching Thumbnail.
Source: powerfill_switching_thumbnail.srd filters to pool_action = 'Swapped In' specifically, and UI labels elsewhere use Remaining / Leaving / Joining / Switching.
Confidence: High. Pattern is clear.
If wrong: Our port might not produce the Swapped In internal state, breaking the Switching Thumbnail report.
Needs confirmation from Tom/Greg: Are there other internal pool_action values we haven't seen? Is Swapped In paired with a corresponding Swapped Out or just Leaving?
A4: Securitization Rules are Hard Constraints
Claim: If a loan violates a pool's securitization rule (from pscat_securitization_rules referenced via pfill_constraint_sec_rule_rel), it is excluded entirely from candidacy. No exceptions, no warnings, no scoring penalty.
2026-04-16 correction (Phase 2 planning, Architect): Original text cited pscat_securities_sec_rules, which does not exist in the legacy NVO or in the pssaas-db seed schema. The authoritative upstream table is pscat_securitization_rules (singular). Third Phase-0 Truth Rot instance; caught by the Primary-Source Verification Gate.
Source: T-SQL candidacy queries appear to JOIN against securitization rule tables and WHERE-clause-filter on rule satisfaction. This is filter semantics, not scoring.
Confidence: High. This matches agency delivery requirements (Fannie/Freddie/Ginnie pool eligibility is binary).
If wrong: We'd incorrectly allocate loans to pools they're ineligible for, causing downstream delivery failures.
Needs confirmation from Tom/Greg: Are there any sec rules that are soft (warning, not exclusion)? Are there overrides?
A5: Lockdown is Absolute
Claim: Pools/trades flagged with pfill_lockdown_guide.lock_pool = 'y' are completely untouched by the next PowerFill run. Loans in locked pools stay where they are. Locked trades don't receive new loans.
Source: d_maint_lockdown_guide.srd has a lock_pool column with values y/n. T-SQL references to lockdown appear early in psp_powerfill_conset (before main allocation), suggesting lockdown is a preprocessing step that removes locked entities from the working set.
Confidence: High.
If wrong: Our port might re-allocate loans in locked pools or fill locked trades, contradicting user intent.
Needs confirmation from Tom/Greg: Is partial lockdown possible (e.g., lock this pool for settlement Y but not settlement Z)? What if a lockdown conflicts with a sec rule?
A6: Pre-Process is Independent of Main Run
Claim: The menu's "Pre-Process" action runs pre-processing procedures without running the main allocation. Users can Pre-Process repeatedly before running the main PowerFill.
Source: m_powerfill.srm has distinct ue_perform_preprocess and ue_perform_powerfill menu items calling separate events.
Confidence: High.
If wrong: Our API should combine preprocess + main into a single endpoint.
Needs confirmation from Tom/Greg: In practice, do users ever pre-process without immediately running? What's the purpose — to review the working set before committing?
2026-04-16 correction (Phase 3 planning, Architect; Primary-Source Verification Gate): Original text listed five procedures invoked by "Pre-Process" (psp_pfill_bx_cash_grids, psp_pfill_bx_settle_and_price, psp_add_to_pool_lockdown_guide, psp_pfill_ect_params, psp_pfill_trade_params). This was incorrect. The Desktop App's w_powerfill.srw::ue_perform_preprocess event (lines 194-290) invokes only three:
psp_add_to_pool_lockdown_guide(line 217)psp_pfill_ect_params(line 240; comment: "Run the payups procedure")psp_pfill_trade_params(line 263; comment: "Run the extended trade level constraints procedure")
The two BX procedures are invoked from the separate ue_perform_powerfill event (lines 94-192, the main Run event):
psp_pfill_bx_cash_grids @as_price_floor=<...>(line 111)psp_pfill_bx_settle_and_price(line 131)
followed by psp_powerfill_conset (line 147) and psp_powerfillUE (line 171).
Implication for the PSSaaS port: the Phase 3 POST /api/powerfill/preprocess endpoint implements only the 3 ue_perform_preprocess procs. The BX procs move to Phase 6 (Run), where they have access to the bx_price_floor Run option and to run-scoped upstream data (pscat_inst_dde_links_multi, rmcat_setup_risk_parameters, etc.) that they require. This is F2 in the Phase 3 plan's Primary-Source Verification findings. Fourth Phase-0 Truth Rot instance; caught by the Primary-Source Verification Gate.
A7: Priority Iteration Order is Ascending Numeric
Claim: pfill_constraints priority orders allocation: constraints with priority = 1 run before priority = 2, etc. Lower number = higher precedence.
Source: Pattern is common in priority-ordered rule systems and is reinforced by constraint priority modal w_maint_pfill_constraint_order.srw where users reorder rules top-to-bottom.
Confidence: High.
If wrong: Higher-priority constraints wouldn't get first allocation chance, reversing user intent.
Needs confirmation from Tom/Greg: How are priority ties broken? By insertion order, by ID, alphabetical?
A8: pfill_powerfill_guide and pfill_pool_guide Serve Different Purposes
Claim:
pfill_powerfill_guide— allocation output — one row per (loan, trade) recommendationpfill_pool_guide— pool action summary — one row per loan with itspool_actionvalue
Source: pfill_powerfill_guide is referenced in guide reports that include trade details; pfill_pool_guide is referenced in pool-action reports (switching thumbnail, pool candidates) that include pool disposition.
Confidence: Medium. The two tables might be redundant in some cases, or one might be derived from the other.
If wrong: Our port could simplify these to a single table, or vice versa.
Needs confirmation from Tom/Greg: Are these genuinely different data or views of the same? Is one derived from the other?
A9: pfill_cblock_guide is "Contiguous Block" Related
Claim: The cblock in pfill_cblock_guide and pfill_trade_cblock_base refers to some kind of contiguous block allocation — possibly related to agency delivery requirements where pools must contain contiguous pool numbers or blocks of loans.
Source: Pure naming inference. No explicit documentation.
Confidence: Low. Could be "contiguous block," "cash block," or something else entirely.
If wrong: Our port might misunderstand this table and produce wrong output, or over-engineer handling for a pattern that doesn't exist.
Needs confirmation from Tom/Greg: What does cblock actually mean? What's the real business purpose of these tables?
A10: Kickouts are Loans That Couldn't Be Allocated, With Reasons
Claim: pfill_kickout_guide_01 contains loans that the allocation engine tried to place but couldn't, along with a reason code indicating why.
Source: Naming convention matches industry usage ("kickout" is common secondary-marketing vocabulary for a loan rejected from a pool).
Confidence: Medium. The "_01" suffix is unexplained — could imply multiple levels of kickouts, or just a versioning hack.
If wrong: Our port might miss kickout semantics, or misclassify successful allocations as kickouts.
Needs confirmation from Tom/Greg: Why the "_01" suffix? Is there a pfill_kickout_guide_02? What reason codes exist?
A11: Carry Cost Curve Lookup and Computation Formula (VERIFIED 2026-04-16)
Claim: pfill_carry_cost is a lookup table keyed by (investor_instrument_name, on_day) with a to_day upper bound and an annual_rate for each day-bucket. The runtime computation per (loan, trade) candidate is:
avg_rate := AVG(pcc.annual_rate)
over rows where pcc.investor_instrument_name = candidate.market
AND candidate.interest_earning_days BETWEEN pcc.on_day AND pcc.to_day
AND pcc.annual_rate IS NOT NULL
carry_cost := round_NUMERIC(9,6)( avg_rate * interest_earning_days / 365 )
prx_plus_carry := price + (mode == 'po' ? 0
: (note_rate * interest_earning_days / 365) - carry_cost)
Day-bucket lookup is closed inclusive [on_day, to_day] (T-SQL BETWEEN semantics). No interpolation between buckets. Days outside any bucket → NULL carry_cost. The /365 divisor preserves decimal precision because interest_earning_days is declared NUMERIC(4,0) in every legacy temp table that calls the formula (NVO lines 216, 244, 437); T-SQL datatype precedence promotes NUMERIC(4,0) / INT to NUMERIC.
Source (verified Phase 5): NVO n_cst_powerfill.sru lines 1332-1364 (within psp_powerfill_conset) and 14429-14472 (within psp_powerfillUE). Phase 5 plan §3 documents the algorithm with full citations. Empirical reproduction against pssaas-db of the literal NVO line 1361-1364 expression with realistic types (price=99.5, note_rate=6.25, days=15 NUMERIC(4,0), carry_cost=0.014795, mode='pc') returns prx_plus_carry = 99.742049, confirming the net-interest term contributes the expected ~0.2568 and is NOT zeroed by integer truncation.
Confidence: High on the formula and types (verified line-by-line + empirical SQL reproduction). Medium on rounding-mode parity for the NUMERIC(9,6) cast (Phase 5 implements MidpointRounding.AwayFromZero to match T-SQL CAST behavior; Phase 9 parallel-validation will confirm or surface a delta).
If wrong: Our carry cost computation would diverge from Desktop App. Phase 9 parallel-validation tests for this.
No longer needs Tom/Greg confirmation on the formula. Remaining items for Tom/Greg: (1) intent on 'any' row in pfill_carry_cost — wildcard or literal? (Phase 5 implements literal-match per NVO; flagged as future product enhancement); (2) intent on rounding-mode parity vs. banker's rounding for new code.
Phase 0 framing of "days × rate" was incomplete — it omitted the /365 day-count divisor and the dual-mode prx_plus_carry formula with its net-interest correction. Caught by Phase 5 Primary-Source Verification Gate finding F1.
A12: psp_pfill_bx_cash_grids is Optional
Claim: The BestEx cash grids procedure is only called if the bx_price_floor option is set. Otherwise, it's skipped and the run proceeds without price floor filtering.
Source: w_powerfill.srw ue_perform_powerfill event logic checks if price floor is set before calling the procedure.
Confidence: High.
If wrong: Our port might always call it, introducing unnecessary work.
Needs confirmation from Tom/Greg: Any scenarios where we'd want to force-refresh cash grids regardless of floor setting?
A13: Run Concurrency is Unmanaged (Desktop App)
Claim: The Desktop App assumes single-user PowerFill runs — no locks on pfill_* tables. Two users running simultaneously would produce undefined output.
Source: No locking constructs observed in the procedure text. Desktop App's single-user model implies this was never a concern.
Implication for PSSaaS: We must add run-concurrency control (per BR-8, one run per tenant at a time). This is an enhancement, not a port deviation.
Confidence: High for the legacy behavior; the PSSaaS change is explicit in the spec.
A14: Scoring Formula = Base Price + Feature Adjustments ± Carry
Claim: The loan-trade score is computed as price + feature_adjustments (for PriceOnly mode) or price + feature_adjustments - carry_cost (for PricePlusCarry mode).
Source: Column naming (prx_and_carry vs price), "Price only" vs "Price + Carry" option values in d_powerfill_options.srd.
Confidence: Medium. The exact formula — especially sign conventions for carry — is reverse-engineered.
If wrong: Scoring would produce the same rankings with different absolute values, OR it could produce different rankings if the sign is wrong.
Needs confirmation from Tom/Greg: Is carry subtracted or added? Are there other components (e.g., BUBD fees, SRP) in the score?
A15: Loan Stages are Status-Based Filters
Claim: pscat_loan_stages filtering excludes loans that haven't reached a minimum status (e.g., "Closed") specified in the run options. This is applied as a candidacy filter, not a post-allocation check.
Source: Options include min_status and T-SQL references pscat_loan_stages in candidacy queries.
Confidence: High.
If wrong: We'd either include ineligible loans or exclude eligible ones.
Needs confirmation from Tom/Greg: Is the "min status" strict (must equal) or hierarchical (must be at-or-above)? What's the stage hierarchy?
A16: Trade Tolerance is Symmetric
Claim: Trade tolerance bands apply equally to over-fill and under-fill. A trade with target $5M and tolerance $50K accepts fills between $4.95M and $5.05M.
Source: Single tolerance_amount column in pfill_pool_guide and pscat_trades (not separate over/under columns).
Confidence: Medium-High.
If wrong: We might allow asymmetric tolerance that the Desktop App rejects.
Needs confirmation from Tom/Greg: Any trades with asymmetric tolerance?
A17: Run Output Can Be Re-Queried After the Fact
Claim: After a run completes, the output pfill_* tables persist until the next run. Users can navigate away from the Guide tab and return to see the same results.
Source: DataWindows refresh from the tables on demand; no evidence of in-memory-only state.
Confidence: High.
If wrong: Our API design would need to cache results differently.
A18: Existing Pool Disposition is a View, Not a Table
Claim: pfillv_existng_pool_disposition (note the v prefix for "view") is a SQL view that joins pfill_pool_guide with pool and loan data. Not a physical table.
Source: Naming convention (pfillv_* vs pfill_*) and usage in powerfill_existing_pool_disposition.srd.
Confidence: High.
If wrong: Our port would replicate view logic as a materialized table unnecessarily.
A19: Pre-Session vs Post-Session in Recap
Claim: "Pre-Session" and "Post-Session" in the recap report (d_powerfill_recap) refer to before PowerFill run and after PowerFill run respectively, not to market sessions or settlement sessions.
Source: Column meanings inferred from the recap's purpose (showing trade fulfillment improvement from the run).
Confidence: Medium-High.
If wrong: The recap semantics would be different (e.g., morning session vs. afternoon session).
Needs confirmation from Tom/Greg: Confirm the session vocabulary matches "run" in our spec.
A20: BestEx Integration is Read-Only from rmcat_bx_setup_instr_inv
Claim: PowerFill reads BestEx configuration from rmcat_bx_setup_instr_inv to determine which investor instruments map to which trade instruments. It does not modify BestEx data.
Source: Read-only references in candidacy queries.
Confidence: High.
If wrong: PowerFill could be modifying BestEx setup, which would have side effects on BestEx results.
A21: Embedded Pay-Up Cash Instrument (EPCI) Parameters
Claim: pfill_epci_params stores per-instrument embedded pay-up cash parameters — "EPCI" best guess: "Embedded Pay-up Cash Instrument" — used for cash-grid trades that have pay-up components. Keyed by cash_payup_instrument, with geographic/occupancy/loan-amount/LTV/FICO filter bounds.
Source: NVO n_cst_powerfill.sru line 6094 creates the table as pfill_epci_params (NOT pfill_ect_params as earlier assumption-log drafts claimed — that was a mis-read). Columns: cash_payup_instrument (PK), last_ed, prop_state, occupancy, lower_loan_limit, upper_loan_limit, lower_ltv_limit, upper_ltv_limit, lower_fico_limit, upper_fico_limit.
Confidence: Low on the meaning of "EPCI"; high on the column structure (transcribed from DDL). The companion table pfill_payups (line 6537) holds the actual pay-up pricing grid; pfill_epci_params appears to hold per-cash-payup-instrument eligibility filters.
If wrong: Our port could miss critical cash grid pay-up logic.
Needs confirmation from Tom/Greg: What does "EPCI" stand for? What's the relationship between pfill_epci_params (per-instrument filters) and pfill_payups (pricing grid) and rmusr_payups (user-customizable Risk Manager pay-up data)?
2026-04-16 correction (Phase 1 planning, Architect): This assumption was originally titled "Embedded Pay-Up Cash (ECT) Parameters" and cited pfill_ect_params. That was incorrect — the legacy NVO uses pfill_epci_params throughout for the table. The correction triggered the "Phase-0 Truth Rot" antipattern nomination.
2026-04-16 amendment (Phase 3 planning, Architect; Primary-Source Verification Gate): The Phase 1 correction above over-generalized. The procedure that populates pfill_epci_params is named psp_pfill_ect_params (NOT psp_pfill_epci_params). Evidence:
n_cst_powerfill.sru:6890—ls_sp = "psp_pfill_ect_params"(registration)n_cst_powerfill.sru:6909—CREATE PROCEDURE dbo.<ls_sp> WITH ENCRYPTION AS(resolves todbo.psp_pfill_ect_params)n_cst_powerfill.sru:6911— procedure body:INSERT INTO dbo.pfill_epci_paramsw_powerfill.srw:240— Pre-Process event invokespsp_pfill_ect_params
The Phase 1 table rename did NOT apply to the procedure name; the Phase 3 kickoff prompt mis-propagated it. Per ADR-006 schema preservation, PSSaaS ports the procedure with its legacy name (psp_pfill_ect_params) even though it inserts into pfill_epci_params. Fifth Phase-0 Truth Rot instance; caught by the Primary-Source Verification Gate.
A22: Cash Market Map is a Run-Scoped Join
Claim: pfill_cash_market_map is rebuilt each run by joining loan instruments, BestEx setup, and cash grid configuration. It's ephemeral working data, not persistent.
Source: The table is cleared at the start of each run (patterns consistent with working tables).
Confidence: High.
If wrong: Our port might treat it as configuration when it's actually runtime data.
A23: "Phantom" Trades May Be Excluded
Claim: pscat_trades.phantom_indicator may flag placeholder/phantom trades that PowerFill excludes from allocation.
Source: Column exists in pscat_trades but not observed in PowerFill's candidacy queries. Inferred exclusion.
Confidence: Low-Medium.
If wrong: We might allocate to phantom trades that shouldn't be filled.
Needs confirmation from Tom/Greg: Does PowerFill respect phantom_indicator? What is a phantom trade?
A24: Pair-Offs Aren't Fed to PowerFill
Claim: pscat_pair_offs (1,513 rows in PS_DemoData) represents trade pair-offs that have already been resolved. PowerFill ignores pair-offs entirely — they're completed trade matches, not allocable capacity.
Source: No direct references to pscat_pair_offs in candidacy queries.
Confidence: Medium.
If wrong: We could over-count trade capacity.
Needs confirmation from Tom/Greg: Do pair-offs affect PowerFill at all?
A25: Run Cadence is On-Demand
Claim: Users run PowerFill interactively, typically 1-3 times per day, in response to pipeline/trade changes. Not scheduled.
Source: UI-driven (menu action in PB window), no scheduling constructs observed.
Implication for PSSaaS: We can add scheduled runs as an enhancement, but the default execution model is on-demand.
Confidence: High.
A26: Phase 1 Accepts CHAR↔NVARCHAR and DATETIME↔datetime2 Drift
Claim: The PSSaaS Phase 1 EF Core entities map CHAR(n) legacy columns to NVARCHAR(n) and DATETIME legacy columns to datetime2. The underlying pfill_* tables still use legacy types; EF reads and writes against them via type coercion. Parity behavior is expected for Phase 1's scope (schema deployment + unit tests + integration test for round-trip) but has not been validated end-to-end against a customer pipeline.
Source: PowerFill Phase 1 plan (2026-04-16), Open Question #3. Kevin approved "Option (a) — accept NVARCHAR for Phase 1, log as drift" when approving the plan as specified. The PSSaaS integration test SchemaScriptIntegrationTests.SchemaScript_DeploysCleanly_AndModelMatchesActualSchema treats these drift categories as known-equivalent and emits diagnostic output rather than failing.
Confidence: High on the drift's existence; Medium on its operational safety. SQL Server coerces implicitly in most scenarios, but trailing-space semantics for CHAR and precision differences for datetime2 may manifest in parallel-validation deltas.
If wrong: Parallel validation against the Desktop App (Phase 9) will show output deltas that trace to trailing-space comparisons or sub-second precision differences. Remediation is a Phase 9 task: add [Column(TypeName = "char(n)")] plus a trim value-converter for string properties, and [Column(TypeName = "datetime")] for DateTime properties. These are mechanical once we know the fix is needed.
Needs confirmation from Tom/Greg: Any known scenarios where CHAR trailing-space semantics matter to PowerFill output? Any downstream procedures that rely on DATETIME's lower precision or date-only behavior?
A27: psp_pfill_trade_params CTE Embeds Tenant-Specific Instrument Defaults
Claim: The T-SQL body of psp_pfill_trade_params (NVO n_cst_powerfill.sru lines 5900-5999) opens with a CTE cte_default_inst containing ~60 UNION rows that hardcode specific instrument names with their default upper_loan_limit, upper_fico_limit, occupancy, and prop_state values. Examples: '15 fnma cash 85k' → upper loan 84999.99, '15 fnma cash NY' → prop_state 'NY'. The CTE is joined in the INSERT to populate pfill_trade_params for trades whose instrument matches a listed name.
Source: NVO lines 5902-5960 (the CTE body); join at line 5975.
Confidence: High on the observation (direct transcription); Low on the intent — these instrument names read like a specific customer's setup, not a product-wide reference list.
If wrong (i.e., if these are product-universal): Our port just inherits the correct behavior.
If wrong (i.e., if these are tenant-specific): Different tenants have different instrument names; the hardcoded CTE would produce wrong defaults or fail to match any trade's instrument. In the worst case, pfill_trade_params would stay empty and Phase 6 allocation would have no per-trade bounds.
PSSaaS Phase 3 treatment: Port verbatim per ADR-006 (schema preservation). Flag for Phase 9 modernization: likely the CTE should become a reference table (pfill_instrument_defaults or similar) populated per-tenant at provisioning time.
Needs confirmation from Tom/Greg: Are these instrument names product-universal, or are they from a specific customer's Desktop App configuration? What's the intended mechanism when a new tenant adds a custom instrument name?
Live-System Findings (2026-04-16, PS_DemoData Probing)
The following five assumptions/findings were discovered when the Collaborator first probed PS_DemoData with kevin_pssaas_dev credentials, attempting to deploy 001_CreatePowerFillSchema.sql and 003_CreatePowerFillProcedures.sql. They constitute a Primary-Source Verification Gate finding-set, with explicit Gate Output Action dispositions per process-discipline v4.1.
A28: PS_DemoData Has 3 pfill_syn_* Synthetic-Trades Tables Missing From Spec (RESOLVED 2026-04-19 by Phase 6d empirical DDL recovery + deploy)
Claim: PS_DemoData contains three tables not in our Phase 1 reverse-engineering: pfill_syn_powerfill_guide (40 cols), pfill_syn_powerfill_guide_all_rank (40 cols), pfill_syn_trade_base (11 cols). All three are referenced extensively in the legacy NVO at lines 8950, 11170-11180, 13444-13451, 19019-19370 — the synthetic-trades subsystem of the allocation engine. They are populated and consumed by procedures that live OUTSIDE n_cst_powerfill.sru::of_update_database (which Phase 1 used as the source of truth for the 17-table count).
Source: Live PS_DemoData query (2026-04-16):
information_schema.tables WHERE table_name LIKE 'pfill[_]syn%' → 3 rows
Also referenced in n_cst_powerfill.sru synthetic-trades logic block (line 19019: /*Logic for synthetic trades*/).
Confidence: High on existence; Low on full semantics (we haven't traced the population path).
Resolution (2026-04-19, Phase 6d):
The Phase 6d Architect:
- Read the syn-trades deep dive (Phase 0 prep work).
- Re-probed PS_DemoData
information_schema.columnsfor the 3 tables (40+40+11 cols verified). - Re-probed PS_DemoData
sys.indexesfor PK shapes (per F-6d-8, see A55 below):pfill_syn_trade_basePK =(syn_trade_id, rate)CLUSTEREDpfill_syn_powerfill_guidePK =(loan_id)CLUSTEREDpfill_syn_powerfill_guide_all_rankno PK (heap; ranked candidate set)
- Empirically corrected the syn-trades deep dive's column-type hypothesis (F-6d-3): the deep dive said
pfill_syn_powerfill_guide.syn_trade_idwas the onlyintcolumn. PS_DemoData reality: ONLYpfill_syn_trade_base.syn_trade_idisint(10,0) IDENTITY. Bothpfill_syn_powerfill_guide.syn_trade_idANDpfill_syn_powerfill_guide_all_rank.syn_trade_idarevarchar(30) NOT NULL. (Deep dive's Medium-confidence flag was warranted.) - Deployed via
010_CreatePowerFillSynTradesSchema.sql(4 CREATE TABLE blocks with idempotent guards + PRINT-in-guards per A32 + A50 SET preamble for consistency with 008/009). Empirical PS_DemoData column types are the DDL source of truth. - Added 4 EF Core entities (
PowerFillSynTradeBase/PowerFillSynPowerFillGuide/PowerFillSynPowerFillGuideAllRank/PowerFillPowerFillLog) registered inPowerFillModule.cs. The_all_rankentity is registered as keyless (HasNoKey()) per F-6d-8. - The 3 syn tables +
pfill_powerfill_logare populated bypsp_powerfillUE(NVO 13246-19801; deployed via011_CreatePowerFillUeProcedure.sql). UE runs synthesis logic at NVO 19019-19347 (first pass) + 19450-19790 (second pass for unallocated loans) + the recommended-synthesis filter (Phase E in deep dive).
Phase 6d PoC against PS_DemoData: The 3 syn tables are not actually populated because the 6c A54 PK violation in psp_powerfill_pool_guide fires at Step 5 (BEFORE Step 6/UE runs). Per Option C disposition (PO-confirmed), this is documented expected behavior; Phase 9 parallel-validation against a clean Desktop App customer DB is the gate.
The "17 tables" Phase 1 count is now corrected to "20 tables + pfill_powerfill_log = 21 PowerFill-owned tables" (17 from 001 + 3 syn + 1 log from 010).
Confidence: Resolved. Empirical DDL recovery + deploy verification + EF entity registration. Tom/Greg input remains optional (the 5 Open Questions in the syn-trades deep dive § "Open Questions for Tom/Greg" are still open at the semantic level — e.g., why DELETE not TRUNCATE, why 4 parallel rank columns, what assign codes mean).
Needs confirmation from Tom/Greg (DEFERRED — not blocking):
- Why DELETE not TRUNCATE on the 4 run-output tables (NVO 13441-13451) but TRUNCATE on
pfill_syn_trade_base(NVO 19020)? - Why 4 parallel rank columns? Are all used or vestigial?
- What does the
assigncode vocabulary fully mean (y/o/ox/ycz/ypl/Ox)? - Is there a configuration toggle to disable synthesis?
- Is
pfill_powerfill_logconsumed by anything (forensic UI, log shipper, etc.) or is it write-only diagnostics?
A29: PS_DemoData Schema Uses VARCHAR(N), Not CHAR(N) (Confirms A26 in the Opposite Direction)
Claim: Every CHAR(N) column declared in 001_CreatePowerFillSchema.sql exists in PS_DemoData as VARCHAR(N) with the same length. Confirmed for all 17 expected pfill_* tables via information_schema.columns query. A26 anticipated drift in the form CHAR → NVARCHAR; the actual drift is CHAR → VARCHAR — same length, no Unicode widening, but trailing-space semantics differ.
Source: Live PS_DemoData query (2026-04-16). Schema dump at ~/.cursor/projects/.../agent-tools/44339f40-...txt.
Confidence: High.
Gate Output Action — Deferred with justification. Phase 4 CRUD work targets the local pssaas-db container which uses our 001 schema (CHAR). PS_DemoData parity validation is Phase 9 scope. The Phase 9 hardening pass per A26 should now plan for [Column(TypeName = "varchar(n)")] mappings (not char(n)), and decide whether to migrate the local pssaas-db schema to VARCHAR to match production semantics.
Implication for Phase 6 design: EF queries that compare strings against PS_DemoData columns (e.g., WHERE investor_id = @id) will behave differently against VARCHAR (no padding) than against CHAR (right-padded to length). Test fixtures built against pssaas-db may pass while production queries fail or vice versa.
A30: kevin_pssaas_dev DDL Permissions on PS_DemoData (RESOLVED 2026-04-16 — option (b) chosen)
Original claim (morning of 2026-04-16): The kevin_pssaas_dev SQL user has db_datareader + db_datawriter + CONNECT on PS_DemoData and nothing else. It cannot CREATE PROCEDURE, CREATE TABLE, CREATE VIEW, or ALTER any object. Confirmed by direct attempt: deploying 003_CreatePowerFillProcedures.sql returned Msg 262: CREATE PROCEDURE permission denied in database 'PS_DemoData'.
Source (original): Live PS_DemoData attempt 2026-04-16 morning. Permission verified via sys.database_role_members and sys.database_permissions queries.
Confidence: Certain (at the time).
Resolution (evening of 2026-04-16): Kevin (PO) added db_ddladmin to kevin_pssaas_dev on PS_DemoData via Azure Portal. This is option (b) from the original disposition. Verified empirically:
sys.database_role_membersnow listsdb_datareader,db_datawriter,db_ddladminforkevin_pssaas_dev.- Test
CREATE TABLE dbo._kpd_perm_test (id INT)succeeded;DROP TABLEsucceeded. - Test
CREATE PROCEDURE dbo._kpd_perm_proc_test AS BEGIN SELECT 1 ENDsucceeded;DROP PROCEDUREsucceeded.
Both arms of the Deploy Verification Gate satisfied: role membership confirmed (sys catalog) AND DDL execution proven (Architect's A32 refinement).
Gate Output Action — Corrected in place (resolution recorded; supersedes prior "Scope-changed" disposition). Backlog items #23, #24, #25 (PS_DemoData deployment) are unblocked. PS_DemoData becomes the canonical real-data validation surface for PowerFill PoC demos. Phase 4 was previously scoped local-only; that constraint is lifted for any post-Phase-4 work. The 002 views overwrite remains a separate concern (existing PS_DemoData views are WITH ENCRYPTION-deployed and unreadable; we have permission to overwrite but still don't know what we'd be replacing — see deferred Backlog item).
Lesson (Outcome-Linked Retro): Documenting the original constraint with explicit Gate Output Action + a clear resolution path made this resolution trivial to record. If the disposition had been a vague "TBD" the resolution would have been harder to action and harder to communicate.
A31: PS_DemoData Already Contains Production pfill_carry_cost Data (295 Rows)
Claim: PS_DemoData has 295 rows in pfill_carry_cost from the original demo client. All other 16 expected pfill_* tables are present-but-empty (run-output tables that get populated each PowerFill run).
Source: Live PS_DemoData row-count query (2026-04-16).
Confidence: Certain.
Gate Output Action — Corrected in place. The Phase 1 001_CreatePowerFillSchema.sql script header already documented this case ("Safe to run against PS_DemoData where some pfill_* tables already exist (e.g., pfill_carry_cost with 295 rows)"). The script's IF OBJECT_ID IS NULL guards correctly preserve existing data. No change needed; logged here so it's visible at the assumption layer rather than buried in a SQL file header.
Implication for Phase 4: When the carry-cost CRUD APIs are designed, the first tenant to enable PowerFill against PS_DemoData (when permissions are unblocked) inherits 295 carry-cost rows pre-populated. Phase 4 CRUD design should consider whether to surface those existing rows as "tenant has prior configuration" or treat them as opaque.
A32: An Idempotent DDL Script Can "Succeed" Without Running Any DDL
Claim: When 001_CreatePowerFillSchema.sql was run against PS_DemoData with the read-only kevin_pssaas_dev user, the script returned exit code 0 with no error output. This appeared to indicate successful deployment. In reality, every IF OBJECT_ID IS NULL BEGIN CREATE TABLE END block evaluated false (all 17 tables already exist), so zero CREATE statements ran. A CREATE TABLE permission error never fired because no CREATE was ever attempted. If even one expected table had been missing, 001 would have failed with the same Msg 262 we hit on 003.
Source: 2026-04-16 attempted deployment. The Deploy Verification Gate's arm (c) live-DB probe correctly identified the post-state matched the pre-state (still 17 tables, still 295 carry-cost rows, still no procs from 003) — but identifying that no work was done was not the same as identifying that work succeeded.
Confidence: Certain.
Gate Output Action — Corrected in place. The Deploy Verification Gate's arm (c) needs a refinement: for SQL deployments, the verification must distinguish "the DDL was executed" from "the post-state matches expectations," because an idempotent script with all guards triggering can produce the latter without the former. Practical countermeasure for SQL deployments going forward: before declaring success, compare a structural checksum (e.g., OBJECT_ID() for each expected object plus row counts in seed tables) AND confirm that at least one DDL statement actually executed — either via @@ROWCOUNT after the CREATE block, an explicit PRINT 'Created table X' inside each guard, or a deployment marker row inserted into a pfill_deployment_log table.
This refinement should be relayed to PSX as a PSSaaS-experience contribution to the Deploy Verification Gate canonical text. Tagged for v3.1 nomination.
Implication for Phase 4: The Phase 4 SQL artifacts (any new tables for tenant config) should include a PRINT line inside each IF OBJECT_ID IS NULL guard so deploy logs distinguish "table already existed, skipped" from "table created."
A33: Spec Line 83 ("Active Lockdown Rules") vs Schema — Sec-Rule-Rel Guard Is Authoritative (PO Option A, 2026-04-16)
Claim (verbatim prior spec): powerfill-engine.md originally stated: "Prevent deletion of constraints referenced by active lockdown rules" (requirements §Constraint Management).
Schema fact: pfill_lockdown_guide is keyed by pool_name only and has no constraint_name, no investor_id, and no foreign key to pfill_constraints. There is therefore no join path to implement "referenced by active lockdown rules" without inventing semantics.
PO resolution (2026-04-16, Option A): The spec sentence has been amended so the documented rule matches what the schema can enforce. DELETE /api/powerfill/constraints/... returns 409 Conflict when any row exists in pfill_constraint_sec_rule_rel for that constraint composite key; the client must delete sec-rule associations first. This is now the authoritative delete guard, not an interim substitute. The lockdown-derived guard is TBD pending PowerBuilder UI review (ue_delete_constraint, uo_tab_powerfill_constraints) — if review reveals a real constraint↔lockdown linkage, an additive guard will be specified in a future phase.
Source: Primary-Source Verification Gate finding F3 (2026-04-16); 001_CreatePowerFillSchema.sql table definitions; Architect escalation powerfill-phase4-f3-architect-escalation.md; PO decision recorded in this assumption.
Confidence: High on "lockdown literal is not implementable today"; Medium on whether Desktop App eventually ties deletes to lockdowns via unstated UI logic — TBD pending PowerBuilder review.
Gate Output Action — Corrected in place (PO Option A). Spec line 83 amended to the sec-rule-rel rule. Code matches spec; the prior "spec said X / code does Y" gap is closed. Future PowerBuilder review may add an additional lockdown-derived guard, which would be a new requirement (additive), not a contradiction of A33.
If wrong: If subsequent PowerBuilder review reveals the Desktop App truly blocks deletes based on lockdown state, the current guard is incomplete but not incorrect — an additive lockdown check would extend it without contradicting A33.
Phase 4 race-condition (acknowledged, deferred): the interim guard reads pfill_constraint_sec_rule_rel then deletes the constraint without a transaction wrapping both. Concurrent AddSecRuleAsync can land between the check and the delete. Acceptable for Phase 4 (single-operator config UI). Phase 6 should serialize via a tenant-scoped advisory lock or SERIALIZABLE + retry. Tracked in PowerFillConfigurationService.DeleteConstraintAsync XML doc.
Needs confirmation from Tom/Greg: Does the Desktop App ever block constraint delete based on lockdown rows, and if so, what is the exact rule?
A34: Carry-Cost Sign Convention in prx_plus_carry (VERIFIED 2026-04-16)
Claim: In the legacy prx_plus_carry formula (NVO line 1361-1364, re-emitted at 14467-14472), the loan's net-interest income during the carry period is added to price; the cost-of-funds carry is subtracted. The final composition is price + (note_rate * days / 365 - carry_cost). Net effect on the score:
- A high-coupon loan held longer scores higher (lender benefits from collecting note-rate interest while waiting to deliver).
- A high cost-of-funds environment scores lower (carry erodes the score).
Source: NVO lines 1361-1364 and 14467-14472 — both invocations show identical sign pattern. The legacy 'po' (price-only) mode short-circuits this entire net-carry adjustment to 0, so prx_plus_carry = price regardless of carry curve in 'po' mode. The 'pc' (price-plus-carry) mode applies the net adjustment.
Resolves the open question raised by A14: A14 originally flagged the carry sign as "needs Tom/Greg confirmation." Phase 5 Primary-Source Verification Gate finding F2 verified the sign by reading the active legacy line. The commented-out block at NVO 1355-1360 (which would apply note_rate * days/365 - carry_cost without inner parens) shows the same intent.
Confidence: High on the sign (verified line-by-line + empirical SQL reproduction confirms the net-interest term is computed and contributes ~0.2568 for typical inputs). Medium on the underlying economic rationale — the formula treats carry as a cost-of-funds measure rather than as a financing-curve discount, which Greg should confirm.
If wrong (sign reversed): Loans would score in inverse order to legacy. A high-coupon long-carry loan would rank low instead of high. Phase 9 parallel-validation against Desktop App would catch this immediately.
Phase 5 Architect side-note (Counterfactual Retro): During Collaborator review, a hypothetical concern was raised that (interest_earning_days/365) might integer-truncate to 0 (which would have reduced the 'pc' formula to price - carry_cost, suppressing the net-interest term entirely). Empirical reproduction of the literal NVO expression with interest_earning_days NUMERIC(4,0) = 15 returned 99.742049 (with net-interest contribution) rather than 99.485205 (truncated form). T-SQL datatype precedence promotes NUMERIC(4,0) / INT to NUMERIC regardless of parens; the dividend's type, not the parens, controls truncation. The exchange surfaced a candidate Process Discipline pattern: empirical SQL citations must use the literal expression with the actual production variable types, not synthetic substitutes (see Phase 5 plan §11 retro for full discussion).
Needs confirmation from Tom/Greg: Confirm the economic rationale — is carry intended as a cost-of-funds measure (lender pays to hold the loan), and net-interest as the offsetting income (lender earns from holding)? Are there scenarios where the sign should flip (e.g., negative-carry environments)?
A35: PS_DemoData Carry-Cost Rates Are Uniform Placeholders, Not Production Variation
Claim: PS_DemoData's pfill_carry_cost table contains 295 rows that are characterized by:
- Every row has
annual_rate = 0.270000(uniform; no variation across instruments or day-buckets). - Every instrument has the identical 3-bucket coverage:
[0..15], [16..30], [31..45]. Maximum coverage is 45 days. - Plus one synthetic
'any'instrument with a single bucket[0..45](which the legacy formula matches only literally — see F9 in Phase 5 plan).
This is placeholder/test data, NOT realistic cost-of-funds variation. Real production curves vary annual_rate by on_day (longer carry → higher annual rate, e.g., the NVO's own seed insert at line 6660-6687 uses 0.125 → 0.25 → 0.375 → 0.5 → 1.5 across [0..30] / [31..60] / [61..180] / [181..365] / [366..1095]).
Source: Live PS_DemoData GET /api/powerfill/carry-cost characterized 2026-04-16 (Phase 5 plan §2 finding F8). All 295 rows materialized and inspected.
Confidence: Certain — the data was directly enumerated.
Implication for Phase 5 testing: Integration tests against PS_DemoData (D1, D2, D3 in Phase 5 plan §9.3) prove the calculator runs and returns expected-shape output, NOT that the calculator produces meaningful tenant-specific differentiation. They prove "the calculator can read PS_DemoData carry-cost rows and apply the formula"; they do NOT prove "the calculator produces production-realistic scores."
Implication for Phase 9 validation: Real economic-correctness validation requires running PSSaaS PowerFill against a Desktop App customer's real curves and comparing output. PS_DemoData is insufficient for that comparison because the curves themselves are placeholders. This is consistent with the broader "PS_DemoData proves shape, parallel-validation proves correctness" framing established by Phase 1's A26 and Phase 2's preflight defaults.
Implication for product copy / customer onboarding: When the first real tenant onboards to PSSaaS PowerFill, they will need to upload their own carry-cost curves (probably via the Phase 4 CRUD endpoints, or a future bulk-import tool). The 295 PS_DemoData rows are not a starter dataset for new tenants — they're test fixtures specific to that demo database.
Gate Output Action — Corrected in place (logged here so the limitation is visible at the assumption layer rather than buried in the Phase 5 plan).
Needs confirmation from no one — this is empirical fact, not interpretation.
A37: pfill_powerfill_log Is Auto-Created by psp_powerfillUE, Not Orphaned (RESOLVED 2026-04-19 by Phase 6d explicit deploy)
Claim: The pfill_powerfill_log table is not an orphaned table waiting for an owner — it is auto-created at the top of psp_powerfillUE's body via an IF OBJECT_ID IS NULL guard, then TRUNCATEd on each run, then populated by 6+ INSERT calls during the UE pass.
Source: NVO n_cst_powerfill.sru line 13265-13275 shows the auto-create block (IF OBJECT_ID('dbo.pfill_powerfill_log', 'U') IS NULL ... CREATE TABLE dbo.pfill_powerfill_log). Line 13281 shows the TRUNCATE TABLE pfill_powerfill_log; immediately after. Lines 13555 / 13751 / 13984 / 14527 / 14575 / 14622 / 19080 (and others) show the INSERT INTO dbo.pfill_powerfill_log (log_time, step_name, record_count, details) calls that populate it during UE execution.
Resolves the open question raised by the syn-trades deep dive: powerfill-syn-trades-deep-dive.md §"Open Questions" item #1 asked who owns pfill_powerfill_log and where it gets created. Answer: it's UE-owned, auto-created, populated as a step-by-step run trace.
Confidence: High — direct NVO text inspection plus matching TRUNCATE/INSERT operations. PSSaaS Phase 6d will deploy pfill_powerfill_log explicitly via 007_*.sql to make the dependency visible at DDL deploy time rather than implicit at first run (per ADR-006 schema preservation philosophy: the schema is a deliverable, not a runtime side-effect).
Implication for Phase 6d: The new 007_CreatePowerFillSynTradesSchema.sql must create pfill_powerfill_log alongside the 3 pfill_syn_* tables. The legacy auto-create block in the UE proc body should be preserved verbatim during the port (so that Desktop App customers running both stacks against the same DB don't see "table already exists" errors), but the PSSaaS deploy is what makes the table real.
Resolution (2026-04-19, Phase 6d): Deployed explicitly via 010_CreatePowerFillSynTradesSchema.sql (file numbering is 010 rather than 007 per the breakdown's flexible numbering — 006/007/008/009 were all consumed by 6a/pre-6b/6b/6c respectively). Schema-side: IF OBJECT_ID(N'dbo.pfill_powerfill_log', N'U') IS NULL CREATE TABLE ... matches NVO 13267-13274 verbatim (5-col PK on log_id IDENTITY). Proc-side: the verbatim-port 011_CreatePowerFillUeProcedure.sql preserves the auto-create block at NVO 13265-13275 unchanged, so legacy + PSSaaS can coexist on the same DB without conflict (the auto-create is a no-op when 010 has already deployed the table).
Empirical PS_DemoData state: PS_DemoData did NOT have pfill_powerfill_log pre-deploy (info_schema 0 rows). 010 created it. F-6d-4 (banked observation): PS_DemoData was created by Joe via PowerSeller 7.x → 8.x schema-only upgrade per A44; the table being missing confirms that path skipped the UE proc (which would have auto-created it on first run). Generalizes the A44 family of "legacy-plugin auto-migrations transferred to PSSaaS schema responsibility per ADR-006/021."
Needs confirmation from no one — empirical NVO fact + empirical deploy verification.
A38: psp_pfill_insert4_pool_guide Is a 771-Line Proc Omitted from Phase 6 Kickoff Inventory (RESOLVED 2026-04-19 by Phase 6c empirical NVO trace + deploy)
Claim: The Phase 6 kickoff doc (powerfill-architect-phase6-kickoff.md) listed the proc inventory for Phase 6 work but completely omitted psp_pfill_insert4_pool_guide, a 771-line stored procedure at NVO line 11712-12483.
Source: NVO n_cst_powerfill.sru line 42 declares public function string of_get_psp_pfill_insert4_pool_guide (); line 7035 shows it being executed (if not ln_cst_dbupdate._execute_sql(this.of_get_psp_pfill_insert4_pool_guide(), true) then return false); line 11712 is the body's start (public function string of_get_psp_pfill_insert4_pool_guide ();string ls_syntax).
Phase 6 kickoff verification gate finding F-VERIFY-4 (Architect, 2026-04-17): the proc was discovered while enumerating all public function declarations in n_cst_powerfill.sru. The kickoff doc's NVO citation index (kickoff §"NVO PRIMARY-SOURCE CITATION INDEX") simply did not include this proc.
Confidence: High — three independent NVO references (declaration, execution call, body definition).
Disposition for Phase 6: Sub-phase 6c (pool-action derivation) owns this proc per the Phase 6 sub-phase breakdown (powerfill-phase-6-subphase-breakdown.md). The naming convention (_insert4_pool_guide) strongly suggests it's a sub-step of psp_powerfill_pool_guide. The 6c plan will own the Primary-Source Verification Gate item "where is psp_pfill_insert4_pool_guide invoked from?" before drafting the port.
Implication for process discipline: This is the third Phase-0 Truth Rot finding in Collaborator-authored kickoff docs (along with A39 and A40). The pattern is strong enough to nominate "Reference docs are not primary source" as a candidate refinement to the Primary-Source Verification Gate canonical text. Even Collaborator-authored kickoff documents must be Gate inputs themselves.
Resolution (2026-04-19, Phase 6c empirical NVO trace, finding F-6c-2): The proc is invoked from inside psp_powerfill_pool_guide's body at NVO 11130 (EXEC psp_pfill_insert4_pool_guide) — a runtime call between two INSERT INTO dbo.pfill_pool_guide blocks (NVO 11062 above; NVO 11132+ below). The proc has WITH ENCRYPTION (NVO 11714); takes 0 parameters; spans NVO 11713-12480 (body) inside function 11712-12483.
Disposition: Both procs deployed in 009_CreatePoolGuideProcedure.sql (insert4 first lexically so it's available for runtime EXEC; pool_guide second). No Alternatives-First decision needed — the call site is in the proc body text. Single-file deploy mirrors the legacy two-procs-in-one-PB-function-pair layout.
Empirical confirmation: 009 deploys cleanly to local pssaas-db AND PS_DemoData (OBJECT_ID() returns non-null for both procs on both DBs after deploy). 6c PoC against PS_DemoData reaches Step 5 (pool_guide) successfully and EXECs psp_powerfill_pool_guide — the proc runs for ~31s before failing at NVO 9921 (INSERT INTO ##cte_posting_set_1300, well BEFORE the inner EXEC psp_pfill_insert4_pool_guide at NVO 11130) due to a separate legacy-proc PK design bug surfaced by PS_DemoData snapshot data shape (per A54 below). The A38 resolution itself is verified: the deploy ordering works; the inner EXEC just isn't reached on this snapshot.
Needs confirmation from no one — empirical NVO fact + empirical deploy verification.
A39: psp_powerfill_pool_guide Live Body Is at NVO 8770-11185, Not 7194-11186 (VERIFIED 2026-04-17)
Claim: The Phase 6 kickoff doc (powerfill-architect-phase6-kickoff.md) cited psp_powerfill_pool_guide body at NVO line 7194-11186 (≈3,992 lines). The actual live body is at NVO line 8770-11185 (≈2,415 lines). Lines 7194-8769 are a commented-out older version of the same proc, kept in NVO source as a historical reference but not executed.
Source: Phase 6 kickoff verification gate finding F-VERIFY-5 (Architect, 2026-04-17). NVO inspection between lines 7194 and 8770 reveals each line of T-SQL string literal is wrapped in PowerBuilder block-comment markers (/* ... */), and the _execute_sql call near line 7035 invokes only the latter declaration. PowerBuilder comments out PB code, but T-SQL string literals stored as PB strings can also have entire blocks visually commented out via /* */.
Confidence: High — direct NVO inspection. The kickoff's wider citation reflects either (a) an earlier dump of the file before the 8770 declaration was added, or (b) a mechanical line-range derivation that didn't filter out the PB-commented historical block. Either way, the live body is what Phase 6c ports.
Implication for Phase 6c: The 6c plan ports the live body at NVO 8770-11185 (~2,415 lines). The dead block at 7194-8769 is explicitly excluded from port scope and should be marked as dead_code: true if any inventory tool catalogs it. The plan should not waste cycles on the dead version.
Implication for process discipline: Same as A38 — third Phase-0 Truth Rot finding in a Collaborator-authored kickoff doc.
Needs confirmation from no one — empirical NVO fact.
A40: psp_powerfill_conset Has 6 Parameters, Not the 7 the Phase 6 Kickoff Listed (VERIFIED 2026-04-17)
Claim: The Phase 6 kickoff doc (powerfill-architect-phase6-kickoff.md) listed psp_powerfill_conset as taking 7 parameters including @ad_bx_price_floor. The actual proc has 6 parameters and bx_price_floor is psp_pfill_bx_cash_grids's parameter, not psp_powerfill_conset's.
Source: NVO n_cst_powerfill.sru line 51-58 shows the conset CREATE PROCEDURE signature with exactly 6 parameters:
CREATE PROCEDURE [dbo].[psp_powerfill_conset]
@as_scope varchar(2),
@as_price_value varchar(2),
@as_status_code varchar(30),
@ai_max_eligible_days integer,
@ai_max_trade_settle_days integer,
@ai_eligible_settle_buffer_days integer
WITH ENCRYPTION AS
PB front-end w_powerfill.srw line 154-169 populates exactly these 6 values into ls_proc_parms[1..6] and passes them to both psp_powerfill_conset and psp_powerfillUE (line 170-172). The bx_price_floor value, when set, is passed separately to psp_pfill_bx_cash_grids at line 111 of w_powerfill.srw (per A12 — that proc is skippable when bx_price_floor is unset).
Phase 6 kickoff verification gate finding F-VERIFY-3 (Architect, 2026-04-17): cross-referenced the kickoff's parameter list against the actual proc CREATE statement.
Confidence: High — direct NVO inspection of both the proc signature and the front-end invocation.
Implication for Phase 6a / 6b: Sub-phase 6b's port of psp_powerfill_conset deploys a 6-parameter proc. Sub-phase 6a's BX cash-grid pre-step (psp_pfill_bx_cash_grids) takes bx_price_floor as its own parameter. The PSSaaS run-orchestration code (sub-phase 6a PowerFillRunService) must NOT attempt to pass bx_price_floor to conset — doing so would fail with a "wrong number of parameters" SQL error at runtime.
Implication for the spec: Spec (powerfill-engine.md) §Run Options should be reviewed during sub-phase 6a planning to confirm it doesn't make the same 7-parameter mistake; if it does, the spec amendment for Q9 (defaults drift) should also fix the parameter count.
Implication for process discipline: Same as A38, A39 — third Phase-0 Truth Rot finding in a Collaborator-authored kickoff doc. The Architect Primary-Source Verification Gate caught all three before any code was written, validating the gate's worth at planning-input scope.
Needs confirmation from no one — empirical NVO fact.
A1.0 (PLACEHOLDER): Multi-Pass Allocation Pass Boundaries — TBD by Phase 6b Empirical NVO Trace
Claim (placeholder): Assumption A1 (allocation runs in "four discrete passes: exact fit, best fit, fill remaining, orphan handling") is currently unverified at the pass-boundary level. Phase 6b will trace the actual pass structure from the NVO body of psp_powerfill_conset (NVO line 50-5886; specifically the multi-pass blocks scattered through NVO line 7500-12500 per the kickoff's structural hint) and document each pass as A1.1, A1.2, A1.3, A1.4.
Source: Phase 6 kickoff verification gate (Architect, 2026-04-17) flagged this as a critical-path question for sub-phase 6b. PO accepted Architect default (Option A in powerfill-phase-6-open-questions.md Q4): empirical NVO trace inside 6b's range, with explicit Phase 9 critique hook.
Status: Open — to be resolved by sub-phase 6b plan §3 (Algorithm) drafting.
Confidence: Low at the pass-boundary level (4-pass naming is interpretation, not NVO citation); Medium that some multi-pass structure exists (the conset body is too large to be a single pass, and NVO 7500-12500 contains identifiable separate INSERT INTO @working_table blocks with different WHERE clauses). The empirical trace will reduce both estimates to High once complete.
If wrong (passes are differently structured than 4-fit-types): Sub-phase 6b's allocation engine port may need restructuring; estimate impact ~2-3 days inside 6b's 7-10 day range. Phase 9 parallel-validation against Desktop App is the ultimate safety net.
Closure path: Sub-phase 6b plan §3 documents A1.1-A1.4 (or however many passes exist) with NVO line citations per pass. A1 in this log is updated with a 6b-completion amendment showing the verified passes; A1.0 placeholder is then resolved or removed.
Needs confirmation from Tom/Greg: Optionally — PO may escalate Q4 to Tom/Greg if scheduling permits. Architect default proceeds without this input.
Summary of Open Questions
These items rise to the top and should be the first questions asked when Tom or Greg becomes available:
A1 — exact pass semantics inRESOLVED 2026-04-18 (Phase 6b empirical NVO trace). Structure is 3 stages per constraint inside an outer per-constraint priority loop; original "four discrete passes" labels (exact fit / best fit / fill remaining / orphan handling) do not appear in the NVO and were Phase 0 invention. Canonical labels per NVO comment markers: Setup, Stage 1, Stage 2, Orphan Sweep. Phase 9 parallel-validation is the per-stage semantic correctness gate.psp_powerfill_consetA9 — what "cblock" actually meansRESOLVED 2026-04-18 (Phase 6b A47). cblock = "constraint capacity block": per-trade-per-constraint capacity-exhaustion tracking inside conset. Lifecycle is conset-internal (DELETE at NVO 101-102, INSERT at NVO 5718/5741, LEFT JOIN as eligibility filter at 13 sites across all 3 stages). 6b owns the tables.A11 — carry cost curve columns and interpolation rulesRESOLVED 2026-04-16 (Phase 5 verification). Algorithm verified line-by-line against NVO + empirical SQL reproduction. Closed inclusive interval, no interpolation,/365divisor preserves decimal via NUMERIC(4,0) typing.A14 — exact scoring formula, especially carry signRESOLVED 2026-04-16 (Phase 5, A34). Net-interest income added to price, cost-of-funds carry subtracted;'po'mode short-circuits toprx_plus_carry = price. Greg should still confirm the underlying economic rationale.- A21 — what "EPCI" stands for and what it does (previously mislabelled "ECT"; Phase 3 clarified the procedure name remains
psp_pfill_ect_paramswhile the table ispfill_epci_params) - A23 — phantom trade handling
- A24 — pair-off relevance to allocation
- A27 — whether the
psp_pfill_trade_paramsCTE's ~60 hardcoded instrument names are product-universal or tenant-specific residue - A28 — synthetic-trades subsystem semantics; how/when are
pfill_syn_*populated; what proc(s) own them - A29 (joins A26) — full schema-type drift inventory for Phase 9 hardening; need to decide whether to migrate
pssaas-dbtoVARCHARfor parity - A33 — constraint delete: is blocking on
pfill_constraint_sec_rule_relonly correct, or does the Desktop App enforce a lockdown-derived rule? - A34 (residual) — confirm the underlying economic rationale of the carry sign convention (carry as cost-of-funds, net-interest as offsetting income); confirm sign conventions in negative-carry environments
- A35-related — whether the legacy NVO seed insert (
pfill_carry_costdefaults at NVO 6660-6687:0.125 / 0.25 / 0.375 / 0.5 / 1.5across 5 buckets out to 1095 days) reflects real production-typical curve shape, or is itself placeholder data - 'any' wildcard semantics in
pfill_carry_cost— Phase 5 implements literal-match-only (NVO verbatim). Should'any'become a wildcard fallback as a future product enhancement? - Carry-cost rounding mode — Phase 5 implements
MidpointRounding.AwayFromZeroto match T-SQLCAST(decimal AS NUMERIC(9,6)). Phase 9 parallel-validation will surface any divergence; Greg/Tom may want banker's rounding for non-parity reasons. A1.0 (placeholder) — multi-pass allocation pass boundariesRESOLVED 2026-04-18 (Phase 6b empirical NVO trace). Absorbed into the revised A1 above.A43 — PSSaaS service account EXECUTE permission gap on tenant DBsRESOLVED 2026-04-17 (PO ranGRANT EXECUTE ON SCHEMA::dbo TO kevin_pssaas_devvia Azure Data Studio with theirdb_ownerEntra ID account). Phase 6e tenant-onboarding checklist must still include this step for new tenants.
Additional assumptions may be added as implementation proceeds. Every assumption logged here is explicit about what we're doing and what could be wrong.
A37-A40 Disposition
A37 (pfill_powerfill_log auto-create), A39 (psp_powerfill_pool_guide live body line range), and A40 (conset 6-parameter signature) are all VERIFIED, no open question. They are recorded as fact; consumed by sub-phases 6c (A39) and 6a/6b (A40); and inform a candidate Process Discipline refinement nominated to v3.1: "Reference docs are not primary source — even Collaborator-authored kickoff documents must be Primary-Source Verification Gate inputs themselves." That nomination is tracked separately and is not an open assumption.
A38 (psp_pfill_insert4_pool_guide invocation site) is RESOLVED (2026-04-19, Phase 6c) — invoked from inside psp_powerfill_pool_guide body at NVO 11130; both procs deployed via 009_CreatePoolGuideProcedure.sql. See A38 entry above for the full resolution.
A41-A45 Disposition
A41 (NUMERIC(11,8) param), A42 (BX upstream tables not in seed schema), A44 (legacy plugin auto-migrations general pattern + div_by instance), A45 (mkt_shipped_date 8-table instance) are all VERIFIED + SHIPPED. A41 is encoded in PowerFillRunService typed-decimal parameter. A42 is documented expected behavior (deferred name resolution + structured Failed step). A44 + A45 are shipped in 007_BackfillLegacyPluginMigrations.sql (M1 + M2 respectively). A43 is RESOLVED per the PO's GRANT EXECUTE on PS_DemoData. None require Tom/Greg input.
A46-A51 Disposition (Phase 6b — VERIFIED + SHIPPED)
All six new Phase 6b assumptions are VERIFIED + SHIPPED:
- A46 (kickoff line-range Truth Rot — 7500-12500 is pool_guide, not conset) — corrected in plan §2 + 008 header; 3rd-instance-of-the-pattern process refinement advanced to v3.1 nomination per PO checkpoint.
- A47 (cblock probe disposition: conset-internal lifecycle) — A9 RESOLVED; 6b owns both tables via the verbatim conset port.
- A48 (
@xc / @xcPopMIN-MAX iteration inefficiency) — Phase 9 optimization candidate; not a 6b correctness concern. - A49 (column-missing-on-existing-table CREATE failure mode, distinct from A42) —
008_*.sqldeploys to tenant DBs only; local pssaas-db CREATE failure is documented expected behavior. - A50 (WITH ENCRYPTION proc captures SET QUOTED_IDENTIFIER + SET ANSI_NULLS at CREATE time) — fix shipped in
008_*.sqlpreamble; future PSSaaS proc deploys should follow the same pattern. - A51 (C# candidate-builder and conset have parallel candidate-build logic — observation, not bug) — kept for the diagnostic preview endpoint; Phase 9 parallel-validation determines if outputs diverge meaningfully.
None require Tom/Greg input. All are NVO-empirical or PoC-empirical observations.
A52-A54 Disposition (Phase 6c — VERIFIED + SHIPPED + 1 DEFERRED)
The three new Phase 6c assumptions are VERIFIED + SHIPPED (A52, A53) or VERIFIED + DEFERRED (A54):
- A52 (pool_guide forward dep on
pfill_syn_powerfill_guide— pre-UE snapshot silently matches 0 rows) — (c) Deferred intentionally per breakdown's "6c is the pre-UE snapshot" framing; resolved in Phase 6d's UE pass when synthetic trades populate the table and pool_guide is re-invoked at NVO 19795. 6d update (2026-04-19): UE proc deployed via011_*.sql; UE structurally re-EXECs pool_guide at NVO 19795 per F-6d-7 verification. Practical resolution blocked by A54 — the 6d PoC against PS_DemoData hits A54 in Step 5 BEFORE Step 6/UE runs, so the post-UE pool_guide UPDATEs at NVO 11167-11181 are not yet observable empirically. Phase 9 parallel-validation against a clean Desktop App customer DB is the canonical resolution path. - A53 (BR-3 spec enumerates only 5 of 9 legacy pool_action values; missing Vacated / Ineligible / Swapped Out / Eligible-status-normalization) — (a) Corrected in place via spec amendment in this sub-phase to enumerate all 9 with internal-state callouts.
- A54 (PS_DemoData snapshot triggers latent legacy
##cte_posting_set_1300PK violation; the 2-column PK can't accommodate multi-pa_key loan/trade pairs that the snapshot has) — RESOLVED 2026-04-19 via ADR-021 §Narrow Bug-Fix Carve-Out. Two surgical fixes: (i)_1300PK extension to 3 cols (mirrors_1400); (ii)pt13INNER JOIN qualifier extension (addedAND ps1200_13.settlement_date = pt13.settlement_date). End-to-end Complete run achieved on PS_DemoData (run43e8f148-...). Diagnostic-First Rule revealed the kickoff hypothesis (multi-pa_key Switching) was wrong; the actual fan-out source is settlement-date variance across loans on the same trade. See A54's full RESOLUTION section below + ADR-021 §Narrow Bug-Fix Carve-Out + A65 + A66. (Original deferral text preserved in the HISTORICAL section for the empirical-resolution arc.) 6d update (2026-04-19): PO confirmed Option C at the 6d planning checkpoint (continue verbatim-port discipline; Phase 9 is the gate). A54 fired at Step 5 in the 6d PoC at the IDENTICAL line/loan/trade as 6c (Msg 2627,(36177868, 3385000026)), confirming the data shape is stable. A56 (below) documents the new observation: A54 in Step 5 short-circuits Step 6/UE via the fail-fast contract; the 6d UE proc body was structurally deployed but not yet exercised end-to-end on PS_DemoData.
A38 was previously open and is now RESOLVED (per the A38 entry above) — psp_pfill_insert4_pool_guide is invoked from inside psp_powerfill_pool_guide body at NVO 11130; both procs deployed via 009_*.sql.
A53 and A54 are family-related: both are about legacy proc behavior that interacts with Phase 9 validation (one as a doc-correctness fix; one as a runtime data-shape concern). Both are NVO-empirical + PoC-empirical observations.
Tom/Greg input is optional for A53 (the 9-value classification is direct NVO trace; Tom/Greg could refine the user-vs-internal grouping) and deferred-Phase-9 for A54 (Tom/Greg needed to confirm whether the 2-column PK is intentional and what upstream invariant guarantees no multi-pa_key dups; or whether the PK can be widened in PSSaaS via an ADR-021 amendment for "narrow legacy bug fixes").
A55-A57 Disposition (Phase 6d — VERIFIED + SHIPPED + 1 OBSERVATION)
The three new Phase 6d assumptions are VERIFIED + SHIPPED (A55, A57 OBSERVATION) or VERIFIED + DEFERRED-via-A54 (A56):
- A55 (PS_DemoData empirical PK shapes for the 4 new tables; corrects syn-trades deep dive's column-type hypothesis) — (a) Corrected in place via 010 schema deploy + EF entity registration. PK shapes per F-6d-8:
pfill_syn_trade_base (syn_trade_id, rate),pfill_syn_powerfill_guide (loan_id),pfill_syn_powerfill_guide_all_rankkeyless,pfill_powerfill_log (log_id IDENTITY). Column types per F-6d-3:syn_trade_idisintonly onpfill_syn_trade_base;varchar(30)on the other two syn tables. - A56 (Step 5 fail-fast cascade prevents Step 6/UE from executing on PS_DemoData) — (b) Scope-changed already accounted for — this is the documented Option C outcome PO confirmed at the 6d planning checkpoint. Mathematically equivalent to "A54 fires in 6d UE at NVO 19795 anyway" — even if Step 5's fail-fast were bypassed, UE itself re-invokes pool_guide internally at NVO 19795 and would hit the same PK violation. The doubly-Phase-9-blocked nature is a banked observation, not a port defect. Phase 6e update: A56 carries over unchanged in disposition; 6e's PoC reproduces the IDENTICAL A54 outcome and validates the new orchestration-layer surface (async behaviour, audit row, BR-8, BR-9 cleanup, GET endpoints, cancel) against that predicted outcome. See A56's "Phase 6e carry-over update" sub-section below.
- A57 (kickoff specificity at NVO-line-citation level reduces Truth Rot probability proportionally) — (b) Banked OBSERVATION for the next process-discipline revision. The 6d kickoff's 15+ specific NVO line citations resulted in 0 net-new Truth Rot findings against the kickoff itself (F-6d-1, F-6d-2, F-6d-5, F-6d-6, F-6d-7 all confirmed kickoff claims as exact). Pattern observation: granular NVO citations are self-verifying; vague ranges/hand-waving accumulate drift. Worth feeding into future kickoff drafting practice. Phase 6e update: 2-session corroboration achieved — the 6e prompt is the second consecutive sub-phase with 0 net-new Truth Rot findings against the kickoff. v3.1 nomination drafting is well-supported.
None require Tom/Greg input.
A58 Disposition (Phase 6e — VERIFIED + SHIPPED)
- A58 (BR-9 cleanup scope split: 7 user-facing tables cleared on Failed/Cancelled; 4 syn-trades + log tables preserved for forensics) — (a) Documented in spec + shipped in
PowerFillRunHistoryService.CleanupRunOutputTablesAsync. Empirically verified against PS_DemoData via the 6e PoC (run909d7f16-...). Designed to preserve the forensic value of UE's intermediate output (per 6d D9) — when end-to-end UE runs against an A54-fixed dataset (Phase 9), the syn-trades + log content remains canonical evidence even if the run terminates as Failed.
A41: psp_pfill_bx_cash_grids Parameter Type Is NUMERIC(11,8), Not VARCHAR (VERIFIED 2026-04-17)
Claim: The Phase 6a internal plan (and indirectly the kickoff) initially documented psp_pfill_bx_cash_grids @as_price_floor VARCHAR(...) per "legacy invocation pattern." The actual NVO signature at line 12837 is @as_price_floor NUMERIC(11,8).
Source: NVO n_cst_powerfill.sru:12837 (literal text):
ls_sql = "CREATE PROCEDURE psp_pfill_bx_cash_grids @as_price_floor NUMERIC(11,8) " + &
The PB front-end at w_powerfill.srw:114 casts the call-site value to String(...) only because PB function-argument arrays are typed as String[]. SQL Server then implicitly converts the string back to NUMERIC at EXEC time. Reading the proc signature from the call-site cast was an over-translation.
Confidence: High — direct NVO inspection.
Implication for Phase 6a code: The PowerFillRunService Step 1 invocation passes bx_price_floor as a typed decimal SQL parameter via ExecuteSqlInterpolatedAsync(EXEC dbo.psp_pfill_bx_cash_grids @as_price_floor = {floor}) rather than as a stringified value. This avoids the implicit-conversion round-trip and matches the proc signature.
Process discipline note: This is the fourth Phase-0 Truth Rot finding in a Collaborator-authored kickoff doc / derived plan. Pattern is now strong enough that the candidate refinement "Reference docs are not primary source" should advance from "nominated" to "approved" in process-discipline v3.1.
Needs confirmation from no one — empirical NVO fact.
A42: psp_pfill_bx_settle_and_price Has Hard Upstream Dependencies on rmusr_payups, rmcat_setup_risk_parameters, pscat_comments, pscat_inst_dde_links_multi, pscat_trade_cash_grid (VERIFIED 2026-04-17)
Claim: The Phase 6a BX settle-and-price procedure references several upstream tables that are not in the local pssaas-db seed schema (infra/sql/init/seed-schema.sql). SQL Server's deferred name resolution lets the CREATE PROCEDURE succeed regardless, but EXEC fails at runtime with Msg 208: Invalid object name 'rmusr_payups' against any database missing those tables.
Source: NVO n_cst_powerfill.sru:11345-11710 (proc body), specifically:
rmusr_payupsreferenced at NVO 11362-11365, 11625-11628, 11812-11815, 11830, 11981rmcat_setup_risk_parametersreferenced at NVO 11384, 11389, 11418, 11424pscat_commentsreferenced at NVO 11422pscat_inst_dde_links_multireferenced at NVO 11500, 11503, 11508pscat_trade_cash_gridupdated at NVO 11678-11688
Confidence: High — verified by Grep against NVO body and against seed-schema.sql (zero matches in seed-schema for these table names).
Disposition:
- Local pssaas-db (dev): Step 2 of the synchronous
POST /api/powerfill/runis expected to fail against any tenant whose DB lacks these tables. The PowerFillRunService catches the SqlException and reports it as a step-level failure (the run returns 500 + a structured RunResponse so the operator sees exactly which step + which referenced object failed). This is the documented behavior for local dev, not a bug. Verified empirically against the local API + PSSaaS_Dev DB — error messageSqlException 208: Invalid object name 'rmusr_payups'(run_idf892fbd0-..., 2026-04-17). - PS_DemoData and Desktop App customer DBs: the upstream tables exist (Desktop App writes to them as part of normal pipeline operation), so Step 2 should succeed assuming EXECUTE permission is granted (see 6a-PERM-1 escalation).
Implication for Phase 6 sub-phase planning:
- Sub-phase 6a: orchestration is correct; failure mode is well-defined.
- Sub-phase 6b: does not introduce new upstream dependencies (allocation reads from
pfill_*working tables that 6a/Phase 3 populate). - Future PSSaaS-only modernization: if/when PSSaaS replaces the BX settle-and-price step with a PSSaaS-native pricing source (e.g. PSX-provided BestEx prices), the
rmusr_payupsdependency drops out entirely. Tracked as a Phase 9+ open question, not a 6a concern.
Implication for the local seed schema: Adding rmusr_payups, rmcat_setup_risk_parameters, pscat_inst_dde_links_multi to infra/sql/init/seed-schema.sql would be a dev-experience improvement (lets the local API run a full PowerFill smoke test without an external DB). Not in 6a scope; tracked as a Phase 6e/post-6 backlog item.
Needs confirmation from no one — empirical NVO fact + empirical PoC observation.
A43: PSSaaS Service Account Lacks db_owner on Tenant DBs — Cannot GRANT EXECUTE on dbo Procedures It Created (ESCALATED 2026-04-17)
Claim: The PSSaaS service account on Azure SQL MI tenant databases (kevin_pssaas_dev on PS_DemoData, by way of example) has the db_ddladmin role — sufficient to CREATE the PowerFill stored procedures — but lacks the rights to GRANT EXECUTE on those procedures. As a consequence, after deploying 006_*.sql (or 003_*.sql, 004_*.sql, etc.) to a tenant DB, the API cannot EXEC its own deployed procedures and POST /api/powerfill/run Step 2 fails with Msg 229: The EXECUTE permission was denied.
Source: Empirical observation 2026-04-17 against PS_DemoData:
$ curl -X POST http://pssaas.powerseller.local/api/powerfill/run -H "X-Tenant-Id: ps-demodata" -d '{}'
"error_message": "SqlException 229: The EXECUTE permission was denied on the object 'psp_pfill_bx_settle_and_price', database 'PS_DemoData', schema 'dbo'."
Workarounds attempted:
GRANT EXECUTE ON OBJECT::dbo.<proc> TO <self>— fails withMsg 15281: Cannot grant, deny, or revoke permissions to ... yourselfbecause the deploying principal is the same as the grantee.GRANT EXECUTE ON OBJECT::dbo.<proc> TO public— fails withMsg 15151: Cannot find the object ..., because it does not exist or you do not have permissionbecausedb_ddladmindoes not includeWITH GRANT OPTIONon dbo-owned objects.
Confidence: High — empirical SQL error messages, reproducible via the API.
Pre-existing scope: This is NOT a Phase 6a regression. The same gap affects the Phase 3 preprocess endpoint when invoked against PS_DemoData (the EXECUTE-denied error is identical for psp_add_to_pool_lockdown_guide). The gap was latent because Phase 3-5 PoC work focused on pfill_carry_cost table reads (no proc EXEC), so the surface symptom only emerged when 6a's POST /run was invoked against PS_DemoData.
Disposition: ESCALATED to PO as 6a-PERM-1 (see Phase 6a completion report). The fix is one-time tenant-DB setup performed by a db_owner-class principal, not a SQL deploy script change. Recommended fix:
-- Run as db_owner on each tenant DB (one-time):
GRANT EXECUTE ON SCHEMA::dbo TO kevin_pssaas_dev;
-- OR equivalently: ALTER ROLE db_executor ADD MEMBER kevin_pssaas_dev;
After the grant, the service account inherits EXECUTE permission on all current and future dbo-schema procedures. Phase 6e BackgroundService design must include a tenant-onboarding checklist item for this grant.
Confidence in proposed fix: High — standard Azure SQL MI permission pattern; matches the documented db_executor role from MS Docs.
Needs confirmation from PO: which principal has db_owner on PS_DemoData (likely the original PowerSeller DB account used for snapshot creation), and whether the PO can issue the GRANT, OR whether a brief escalation to the PowerSeller DBA is needed.
A44: Legacy PowerFill-Plugin Auto-Migrations Are a Tenant-DB Schema Concern PSSaaS Must Own (VERIFIED 2026-04-17, RESOLVED for div_by)
Claim: The legacy PowerSeller Desktop App's PowerFill plugin (n_cst_powerfill.sru) performs opportunistic schema migrations on first window-open against any given tenant DB. These migrations live in the plugin source as _set_addcolumn calls inside if NOT f_table_or_view_exists / if not f_table_has_column guards (NVO ~6500-6700). They run silently the first time a user opens the PowerFill window for that DB.
A tenant DB that never had the plugin opened against it (e.g. PS_DemoData, which Joe upgraded from PowerSeller 7.x to 8.x as a schema-only operation per PO confirmation 2026-04-17) will have the base tables but be missing the plugin-added columns. Because PSSaaS inherits the same procedure bodies (per ADR-006 + ADR-021 verbatim port), the procs fail at EXEC time with Msg 207: Invalid column name when they reference these missing columns.
First instance (resolved 2026-04-17): dbo.rmusr_payups.div_by NUMERIC(5,0) NULL — added by NVO line 6577 (initial create branch) and NVO 6592 (the "table exists but missing column" fixup branch). Referenced inside psp_pfill_bx_settle_and_price at NVO 11594, 11602, 11618, 11620, 11649, 11657, 11673. Surfaced as 6a-DATA-1 when POST /api/powerfill/run against PS_DemoData succeeded past the 6a-PERM-1 GRANT EXECUTE wall and hit the next layer:
SqlException 207: Invalid column name 'div_by'.
Invalid column name 'div_by'.
Invalid column name 'div_by'.
Invalid column name 'div_by'.
Resolution path adopted: New PSSaaS deploy script 007_BackfillLegacyPluginMigrations.sql. Idempotent (COL_LENGTH guard); skip-safe when the base table is absent (local pssaas-db per A42). Each migration entry is cited to its originating NVO line and PRINTs deploy-time evidence per A32. Deployed to PS_DemoData 2026-04-17; POST /run then returned status: Complete in 1.63s with all 3 steps succeeded.
Source: Direct NVO inspection (n_cst_powerfill.sru:6563-6596 for the legacy plugin's auto-migration logic; lines 11594-11673 for the proc references). Empirical PoC reproduction: SqlException 207 before 007 deploy → status: Complete after 007 deploy (run_id ae32f0b2-07b6-4a6a-bb5b-b7b25b6f23af, 2026-04-17).
Confidence: High — direct NVO citation + empirical before/after.
Implication for sub-phase planning:
- Phase 6a: orchestration is correct end-to-end; first PoC milestone reached against real customer data.
- Phase 6b: must not surface another
_set_addcolumn-equivalent finding mid-allocation. Recommended pre-6b sweep: an Architect session should enumerate every_set_addcolumncall in NVO 6500-6700 and decide which are 6a/6b/6c/6d-relevant vs Phase 7+-relevant. Each gets a row in007_*.sql(or a successor 008+_*.sql if the file grows too large). This is preparatory work that closes a class of latent finding. - Phase 6e tenant-onboarding checklist: must include
007_*.sqldeployment alongside003-006. Without it, every newly-onboarded tenant DB that hasn't had the legacy plugin run against it will hit the same SqlException 207 cascade.
Implication for the broader port philosophy: ADR-006 and ADR-021 say "port the schema and procedures verbatim." A44 reveals an implicit third concern: PSSaaS inherits the procs but the legacy plugin's schema-migration responsibilities transferred to PSSaaS the moment we own the procedure bodies. This is not a contradiction of ADR-006/021 — it's the natural completion of them. Worth mentioning explicitly in any future ADR about tenant onboarding.
Relationship to existing assumptions:
- A42 (BX upstream tables not in local seed schema) — different concern: those tables don't exist locally. A44 is about base tables that DO exist but have plugin-added columns missing.
- A28 (synthetic-trades subsystem) — same family at a different layer: there
psp_powerfillUEauto-creates entire tables. A44 is auto-creation of columns. Both are PSSaaS-now-owns-this concerns. - A37 (
pfill_powerfill_logauto-create by UE) — same family again; UE auto-creates this table on first invocation. PSSaaS deploys it explicitly in007precedent now generalizes.
Process discipline note: This is the first finding from running POST /run end-to-end. The 5 planning-time + 2 implementation-time Truth Rot findings were all NVO-vs-doc concerns; this is NVO-vs-real-tenant-DB — a third class of finding. Worth folding into the next process-discipline revision as a refinement: "Primary-Source Verification Gate has three layers: NVO-vs-doc, NVO-vs-implementation, and NVO-vs-tenant-DB. Each surfaces different categories of finding; each must be exercised before declaring a sub-phase complete against real data."
Pre-6b sweep update (2026-04-18): A44 explicitly recommended an Architect sweep of every _set_addcolumn call in of_update_database before 6b ships. That sweep was completed 2026-04-18 (completion report). Result: function boundary verified at NVO 6004-6756 (the kickoff's "~6500-6700" was a low estimate); 23 distinct operations enumerated; 21 confirmed out-of-scope (PSSaaS already owns the 17 pfill_* tables via 001, the 2 views via 002, the carry-cost auto-seed is operator-driven per A35, and the pxcat_site_plugins.current_version registry column is Desktop-App-only); only 2 in-scope — M1 rmusr_payups.div_by (already shipped in 007) and M2 mkt_shipped_date DATETIME NULL on 8 base tables (newly shipped in 007 — see A45). Both M1 and M2 are no-ops against PS_DemoData empirically (the sweep verified the columns are already present), but ship as defensive insurance for future tenant DBs that haven't been through the legacy PowerFill window.
Needs confirmation from no one — empirical fact, both before/after.
A45: Legacy Plugin's mkt_shipped_date Migration Spans 8 Base Pipeline Tables, Already Present on PS_DemoData (VERIFIED 2026-04-18)
Claim: The legacy PowerFill plugin's of_update_database function (NVO 6640-6653) iterates 8 base pipeline tables — loan, loan_shipped, loan_modifications, loan_notional, loan_inquiry, psarc_loan_shipped, psarc_loan_modifications, pscat_loan_essential — and adds mkt_shipped_date DATETIME NULL to any that are missing it. This column is referenced by the PowerFill procedure bodies PSSaaS inherits via verbatim port:
psp_powerfill_consetreferences it at NVO 1246, 1450 (AND lls.mkt_shipped_date IS NULLfilter)psp_powerfillUEreferences it at NVO 13851-13853 (in dynamic SQL near 13815)- The Phase 6a PowerFillCandidateBuilder (C# layer) reads it via
Loan.MktShippedDate— Phase 6a already filters candidates withif (l.MktShippedDate is not null) continue(PowerFillCandidateBuilder.csstep 5.1).
A second instance of the A44 family — same shape as the div_by finding (column auto-added by the legacy plugin on first window-open; missing from any tenant DB that was migrated by a non-plugin path).
NVO comment-vs-code drift: the comment at NVO 6640 reads // [TMorgan - 02/07/17] - Add mkt_purchase_date to pipeline tables but the code adds mkt_shipped_date (NVO 6648). mkt_purchase_date is added by some OTHER plugin (BestEx / Risk Manager — out of this sweep's scope; they're separate plugin sources owned by separate workstreams). This drift in the legacy NVO does not affect M2's correctness; the migration we ship adds mkt_shipped_date exactly as the executable code does, not as the misleading comment suggests.
Source: Direct NVO inspection (n_cst_powerfill.sru:6640-6658 for the migration; lines 1246/1450/13851-13853 for the proc references; PowerFillCandidateBuilder.cs for the C# reference). Empirical PS_DemoData state verified 2026-04-18:
$ sqlcmd ... -d PS_DemoData -Q "<8-table mkt_shipped_date check>"
loan, loan_inquiry, loan_modifications, loan_notional, loan_shipped,
psarc_loan_modifications, psarc_loan_shipped, pscat_loan_essential
→ all 8: present
Empirical local pssaas-db state verified 2026-04-18: only loan and loan_shipped exist in the seed schema (both already have mkt_shipped_date); the other 6 tables don't exist locally → skip-safe path fires correctly.
Confidence: High — direct NVO citation + empirical state verification on both DBs.
Disposition: Shipped as M2 in 007_BackfillLegacyPluginMigrations.sql (extended 2026-04-18 by Architect pre-6b sweep). Cursor-driven loop iterates the 8 tables; for each: OBJECT_ID guard for table-missing → PRINT skip; COL_LENGTH guard for column-already-present → PRINT skip; otherwise ALTER TABLE ... ADD mkt_shipped_date DATETIME NULL + PRINT success.
Implication for sub-phases:
- Phase 6b —
psp_powerfill_consetport referencesmkt_shipped_date. The sweep eliminates the latentInvalid column name 'mkt_shipped_date'failure mode that would have surfaced mid-allocation against any tenant DB that didn't have the legacy plugin run on it. Pre-emptive fix; this is exactly what the sweep was for. - Phase 6d —
psp_powerfillUEreferences the column too; same protection. - Phase 6e tenant-onboarding —
007already in the canonical onboarding sequence per A44; M2 is automatically included.
Implication for the spec: None. PowerFill's spec API contracts don't reference mkt_shipped_date directly; it's an internal proc/candidate-builder concern. No spec amendment needed.
Out-of-scope items in of_update_database that were enumerated but NOT ported to 007:
| NVO line | Operation | Reason |
|---|---|---|
| 6037-6042 | pxcat_site_plugins.current_version CHAR(20) | Desktop App plugin-version registry; never used by PSSaaS |
| 6094-6628 | 17 pfill_* _create_table blocks | PSSaaS owns these via 001_CreatePowerFillSchema.sql |
| 6564-6579 | rmusr_payups _create_table (the create branch, not the M1 column-fixup) | Risk Manager's base table; PSSaaS doesn't create it (per A42 it's an upstream dependency); when missing, Phase 6a fails fast with SqlException 208 — documented expected behavior for unseeded local dev |
| 6660-6687 | pfill_carry_cost default-row INSERT when table empty | Phase 4 ConfigurationService manages CRUD per A35 (operator-driven); PSSaaS does NOT seed defaults |
| 6705-6753 | 2 view CREATEs (pfillv_pf_forensics_tradeside, pfillv_existng_pool_disposition) | PSSaaS owns these via 002_CreatePowerFillViews.sql |
The complete classification with NVO line citations is in the pre-6b sweep completion report.
Needs confirmation from no one — empirical NVO + tenant-DB fact.
A1.0 (RESOLVED 2026-04-18 by Phase 6b empirical NVO trace)
Status: RESOLVED. The Phase 6b Architect empirical NVO trace produced the canonical multi-pass structure documented in the revised A1 above (3 stages per constraint inside an outer per-constraint priority loop, contradicting Phase 0's "four discrete passes" interpretation). All A1.0 outstanding items are absorbed into the revised A1.
The placeholder section is retained for historical context (so a future reader following the A1 → A1.0 → revised-A1 thread sees the resolution arc).
A46: Phase 6 Kickoff's "NVO 7500-12500" Pass-Boundary Hint Is Wrong (VERIFIED 2026-04-18)
Claim: The Phase 6 sub-phase breakdown's structural hint that "the multi-pass blocks scattered through NVO 7500-12500" was wrong. The conset proc body is strictly NVO 50-5886 (function of_get_psp_powerfill_conset_syntax); NVO 7192-11185 is the unrelated psp_powerfill_pool_guide (6c territory, per A39).
Source: Direct Grep of ^public function|^end function against n_cst_powerfill.sru produced the function-boundary table; the conset function start is verified at NVO 50, end at NVO 5886. Cross-referenced against A39 (the pool_guide function boundary).
This is the third Phase-0 Truth Rot finding in a Phase 6 reference document — preceded by Phase 6a kickoff F1 (kickoff drift on conset's parameter count) and pre-6b-sweep F-Sweep-1 (kickoff drift on of_update_database line range). PO confirmed in the Phase 6b checkpoint that the candidate process refinement "Reference docs are not primary source — even Collaborator-authored kickoff documents AND Architect-authored sub-phase plans must be Primary-Source Verification Gate inputs themselves" is now 3-session-corroborated and ready to advance from "banked" to "v3.1 nomination."
Confidence: High — direct NVO function-boundary verification.
Disposition: (a) Corrected in place. The Phase 6b plan's §2.4 structural map cites NVO ranges within 50-5886 only; the canonical multi-pass structure (per revised A1) is documented entirely from this verified range. The kickoff doc is historical and does not need amendment (the breakdown's hint was a planning artifact, not a contract).
Process discipline note: The pattern of Phase-0 Truth Rot in derived docs is now strong enough that the Primary-Source Verification Gate canonical text should be updated to enumerate this case explicitly. Banked for next process-discipline revision.
Needs confirmation from no one — empirical NVO fact.
A47: pfill_cblock_guide and pfill_trade_cblock_base Are Conset-Internal — A9 RESOLVED (VERIFIED 2026-04-18)
Claim: Both pfill_cblock_guide and pfill_trade_cblock_base are populated inside psp_powerfill_conset itself, not by any external proc or service. Their full lifecycle is internal to a single conset run:
- DELETE at NVO 101-102 (start of conset, before the constraint loop begins)
- LEFT JOIN as a candidate-eligibility filter at NVO 1589, 1615, 1644, 1692, 1952, 2212, 2824, 3238, 3652, 4260, 4648, 5066, 5372, 5667 (13 read sites scattered across all 3 stages)
- INSERT at NVO 5718 (
pfill_cblock_guide) and 5741 (pfill_trade_cblock_base), both near the end of conset, after the orphan sweep
Resolves A9 — the previously Low-confidence assumption about what "cblock" means. Per the empirical lifecycle, cblock = "constraint capacity block": the proc tracks per-trade-per-constraint capacity exhaustion to prevent over-allocation across stages. When a trade's constraint slot is fully filled inside the proc, a cblock row is written; subsequent stage-2 / orphan-sweep iterations LEFT JOIN cblock to skip already-blocked trades.
Source: Direct Grep of pfill_cblock_guide|pfill_trade_cblock_base against n_cst_powerfill.sru lines 50-5886 (the conset body). 16 references total, all inside conset.
Confidence: High — empirical NVO trace + 16 in-body references.
Disposition for the Phase 6 sub-phase breakdown: (a) 6b OWNS these tables. The breakdown said "if the answer is 'in psp_powerfill_conset itself' then 6b owns them" — that's the answer. No external coordination with 6c/6d/6e needed. 6b's 008_CreateAllocationProcedure.sql deploys the entire conset proc verbatim; the cblock lifecycle deploys with it.
Implication for Phase 7 read APIs: the pfill_cblock_guide rows are observable post-allocation (they persist after conset returns) and are valid output for Phase 7 read APIs. The "pfill_trade_cblock_base" is similarly persistent. Both tables are populated even when no allocation rejects occurred (in the PS_DemoData PoC, both have 0 rows because the data shape didn't trigger cblocks — but the proc wrote 0 rows correctly, not "no INSERT happened").
Needs confirmation from no one — empirical NVO + lifecycle observation.
A48: @xc / @xcPop Constraint Loop Iterates MIN-MAX Range, Not Just Existing Priorities (VERIFIED 2026-04-18)
Claim: The conset proc's outer constraint loop variable @xc is initialized to (SELECT MIN(constraint_priority) FROM dbo.pfill_constraints) at NVO 95, with upper bound @xcPop set to (SELECT MAX(...)) at NVO 94. The loop walks WHILE @xc<=@xcPop and increments by 1 per iteration. If constraint priorities are non-contiguous (e.g., PS_DemoData has priorities 10/20/30), the loop iterates over all integers from MIN to MAX (10..30 = 21 iterations), not just the 3 priorities that exist. Most iterations no-op (no rows match).
Source: Direct NVO 94-95 inspection. The loop body filters by WHERE constraint_priority = @xc so non-existent priorities trivially produce zero rows but still incur the per-iteration setup cost.
Confidence: High — direct NVO inspection.
Performance impact: Bounded by MAX(priority) - MIN(priority) + 1 iterations. PS_DemoData's 10..30 priority range = 21 iterations for 3 actual constraints (~7x overhead vs an iterate-over-existing-priorities loop). On the PS_DemoData PoC, the conset proc completes in ~25s; the inefficiency is not blocking but is observable.
Disposition: (c) Deferred with justification. Phase 9 optimization candidate. NOT a 6b correctness concern — the proc produces correct output; only the elapsed time is suboptimal. ADR-021 verbatim port preserves this behavior; any optimization deviates from the legacy and would require a parity-validation test.
Implication for Phase 6e async runtime: the inefficiency may surface as a noticeable response latency in production (where constraint priority ranges could be much larger — operator-defined). Phase 6e should size async-job timeouts with this in mind.
Needs confirmation from no one — empirical NVO fact + bounded-cost analysis.
A49: WITH ENCRYPTION Procedure CREATE Fails with Msg 207 When Referenced Table Exists But Column Doesn't (VERIFIED 2026-04-18)
Claim: 008_CreateAllocationProcedure.sql deploys cleanly to PS_DemoData but fails CREATE on local pssaas-db with multiple Msg 207, Level 16: Invalid column name '...' errors. The errors reference columns the conset proc body uses that do NOT exist in the local pssaas-db seed schema but DO exist in PS_DemoData (e.g., pscat_trades.rm_trade_type, pscat_trades.tolerance, pscat_trades.designation_date, pscat_trades.rm_loan_hedged_target, pscat_trades.rm_days_required_to_process, loan.orig_loan_amount, pscat_loan_stages.syntax, pscat_trade_cash_grid.sequence_number).
This is a distinct failure mode from A42 (which covers TABLE-missing errors). SQL Server's deferred name resolution lets CREATE PROCEDURE succeed when a referenced TABLE doesn't exist (table is resolved at EXEC time). But when the table EXISTS and a referenced COLUMN doesn't, deferred name resolution does NOT help — the column reference is resolved at CREATE time and the proc creation fails with Msg 207.
Source: Empirical observation 2026-04-18 deploying 008 against local pssaas-db (full sqlcmd output captured in the Phase 6b completion report). The PRINT line at the END of the script DID fire because PRINT is in a separate batch after the failed CREATE — but OBJECT_ID('dbo.psp_powerfill_conset') returned NULL on local pssaas-db, confirming the proc was NOT created.
Confidence: High — reproducible empirical SQL error messages + post-deploy OBJECT_ID check.
Disposition: (c) Deferred with justification. The conset proc's local-pssaas-db CREATE failure is documented expected behavior for unseeded local dev. The proc only needs to deploy successfully on tenant DBs that have the full upstream schema (all PowerSeller customer DBs do; PS_DemoData verified). Local pssaas-db lacks ~6 of the 19 referenced base-schema columns; adding them all to infra/sql/init/seed-schema.sql would be a substantial dev-experience improvement but is OUT of 6b scope.
Implication for Phase 6e tenant-onboarding: new tenants must have a complete base schema (matching PowerSeller 8.x DDL) before 008 deploys cleanly. This is a tenant-provisioning prerequisite documented alongside 6e's BackgroundService scope.
Implication for the local seed schema (deferred Phase 7+ backlog): Adding the missing columns (and probably entire missing tables like pscat_trade_cash_grid) to seed-schema.sql would make local-dev POST /run viable, eliminating one of the differences between local and PS_DemoData PoC behavior. NOT in 6b scope.
Family relationship: A42 (BX upstream tables not in seed schema) + A49 (BX-AND-allocation upstream COLUMNS not in seed schema) + A44 (legacy plugin auto-migrations on existing tables) are three distinct flavors of the same root concern: PSSaaS-local-dev seed schema is a narrow projection of the full Desktop App schema. Phase 9 backlog item: full seed-schema audit + expansion.
Needs confirmation from no one — empirical SQL error + lifecycle behavior.
A50: WITH ENCRYPTION Procedures Capture SET Options at CREATE Time, Not at EXEC Time (VERIFIED 2026-04-18)
Claim: When psp_powerfill_conset was first deployed without explicit SET QUOTED_IDENTIFIER ON; SET ANSI_NULLS ON; preamble, the proc CREATEd successfully but every EXEC from C# (via _db.Database.ExecuteSqlInterpolatedAsync) failed with Msg 1934: SELECT failed because the following SET options have incorrect settings: 'QUOTED_IDENTIFIER'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or filtered indexes ....
The fix attempt of prepending SET QUOTED_IDENTIFIER ON; SET ANSI_NULLS ON; to the C#-side ExecuteSqlInterpolatedAsync SQL did NOT work — the same Msg 1934 still fired. This is documented SQL Server behavior: WITH ENCRYPTION procedures capture the SET options that were active at CREATE time and use those during execution, ignoring session-level SET changes by the caller.
The working fix: prepend SET QUOTED_IDENTIFIER ON; SET ANSI_NULLS ON; GO to 008_CreateAllocationProcedure.sql BEFORE the CREATE PROCEDURE statement. Re-deploy. The proc now binds against the correct SET options at CREATE time, and subsequent EXEC calls succeed regardless of session-level SET state.
Source: Empirical observation 2026-04-18 — three iteration cycle (deploy → EXEC fails Msg 1934 → SET in EXEC fails again → SET in CREATE works). Documented in the Phase 6b completion report PoC verification commands section.
Confidence: High — reproducible empirical evidence + matches documented SQL Server WITH ENCRYPTION semantics (MSDN reference: SET QUOTED_IDENTIFIER (Transact-SQL)).
Disposition: (a) Corrected in place — 008_CreateAllocationProcedure.sql now sets QUOTED_IDENTIFIER ON + ANSI_NULLS ON in a separate batch before the CREATE PROCEDURE block (per A50 finding). C# side does NOT need any SET preamble (and would be a no-op if it tried).
Implication for future PSSaaS proc deploys: The QUOTED_IDENTIFIER ON + ANSI_NULLS ON preamble pattern should be applied to ALL future PSSaaS-deployed procs that query against indexed views, computed columns, filtered indexes, XML data type methods, or spatial indexes. The Phase 6a 003+006 procs may have latent susceptibility to the same issue if the dev's session SET state at deploy-time happened to differ; recommend a sweep of those procs for explicit SET preambles. Banked as a defensive-deploy-pattern observation; not blocking 6b.
Family relationship: This is a deploy-time-vs-EXEC-time semantic of WITH ENCRYPTION, distinct from A49's deploy-time column resolution semantic. Both are caught by Phase 6b's three-layer Primary-Source Verification Gate (NVO-vs-tenant-DB layer surfaces both at CREATE / EXEC time).
Needs confirmation from no one — empirical fact + documented SQL Server behavior.
A51: Phase 6a Candidate-Builder and Phase 6b Conset Proc Have Overlapping Candidate-Build Logic (OBSERVATION 2026-04-18)
Claim: Phase 6a's PowerFillCandidateBuilder (C#) and Phase 6b's psp_powerfill_conset (T-SQL) both build a candidate set internally. The C# candidate-builder writes pfill_loan2trade_candy_level_01; the conset proc has its own setup phase (NVO ~1208-1908) that walks the same pfill_constraints × loan × pfill_trade_base cartesian and writes its own @pfill_loan2trade_candy_level_01 table variable internally. Conset does not appear to read from the C#-written pfill_loan2trade_candy_level_01 table (or it reads it but immediately re-derives the same data anyway).
Empirical evidence: During Phase 6b PoC against PS_DemoData:
- First successful run: candidate_builder reported 1050 candidates; allocation reported 515 allocated.
- Second consecutive run (no data state changes between): candidate_builder reported 0 candidates (because the previous conset run's
DELETE FROM dbo.pfill_trade_baseat NVO 100 wiped the table that 6a reads from); allocation still reported 515 allocated (because conset re-derived its candidate set internally).
If the C# candidate-builder's writes were on conset's read path, the second run would have produced 0 allocations. Instead it produced 515. The C# candidate-builder is therefore NOT on conset's read path.
Source: Empirical PS_DemoData PoC observation 2026-04-18 (run_id captured in Phase 6b completion report).
Confidence: High for the observation; Medium for the interpretation (the C# candidate-builder may serve a separate purpose not visible from this PoC).
Disposition: (c) Deferred with justification. The C# candidate-builder may be:
- Genuinely redundant with conset's internal candidate-build — in which case it's a candidate for removal in a future refactor (would simplify the run service).
- Serving the diagnostic preview endpoint (
POST /api/powerfill/candidates/preview) — the preview endpoint reads candidates without invoking conset, so the C# candidate-builder is the only path for that surface. - Producing a slightly different candidate set than conset's internal logic — Phase 9 parallel-validation against the Desktop App is the gate that determines this.
For 6b, option 2 alone justifies keeping the C# candidate-builder. The preview endpoint is a real Phase 6a deliverable. Removing the C# candidate-builder would break the preview endpoint without an obvious replacement.
Implication for Phase 9 parallel-validation: the validation harness should compare per-loan allocations to the Desktop App, regardless of the C# candidate-builder's intermediate output. The Desktop App doesn't have an equivalent "candidate preview" surface, so divergence in pfill_loan2trade_candy_level_01 between C# and conset is acceptable — what matters is the final pfill_powerfill_guide allocations.
Implication for Phase 7 read APIs: the /runs/{id}/candidates API (Phase 7) should probably read from pfill_loan2trade_candy_level_01 (the C# write target) for the diagnostic-visibility surface, AND from pfill_powerfill_guide for the allocation-decision surface. The two represent different things.
Needs confirmation: None blocking. Phase 9 will surface any divergence in observable behavior.
A52: psp_powerfill_pool_guide Has a Forward Dependency on pfill_syn_powerfill_guide — Pre-UE Snapshot Silently Matches 0 Rows (VERIFIED 2026-04-19)
Claim: The live psp_powerfill_pool_guide body reads from pfill_syn_powerfill_guide at NVO 11171, 11172, 11178, 11180 (4 references inside two UPDATE pg ... INNER JOIN ##cte_detail_0110 ... INNER JOIN pfill_syn_powerfill_guide spg blocks at NVO 11167-11181). pfill_syn_powerfill_guide is populated by psp_powerfillUE (Phase 6d). On a pre-UE PSSaaS deployment (which is what 6c is), the table either doesn't exist (clean DB), is empty (legacy DB without prior PowerFill runs), or contains stale data from a prior Desktop App run.
Behavior: If the table doesn't exist, the proc CREATE succeeds via SQL Server's deferred name resolution; the EXEC fails at runtime with Msg 208: Invalid object name 'pfill_syn_powerfill_guide' ONLY if the UPDATE block is reached. If the table exists but is empty, the INNER JOIN matches 0 rows and the UPDATE silently affects 0 rows.
Source: Direct NVO inspection (n_cst_powerfill.sru:11167-11181 for the UPDATE blocks). Empirical PS_DemoData state verified 2026-04-19: pfill_syn_powerfill_guide table present (legacy schema; 6c PoC didn't reach the UPDATE blocks due to A54).
Confidence: High — empirical NVO + tenant-DB state verification.
Disposition: (c) Deferred with justification. Per the Phase 6 sub-phase breakdown, 6c is the pre-UE snapshot. The breakdown explicitly states: "UE pass / synthetic trades (6d) — UE re-runs the pool-guide step internally and modifies pfill_pool_guide; 6c's output is the pre-UE state." The forward dep is intentional behavior: 6d's UE pass populates pfill_syn_powerfill_guide and then re-EXECs psp_powerfill_pool_guide (per NVO 19795 — verified by Phase 6c plan §2 F-6c-7). The post-UE pfill_pool_guide reflects the synthesis-substituted prices.
Implication for sub-phase 6c PoC: The pool_guide step's RunSummary.pool_guide_count and the BR-3 pool_action distribution reflect the pre-UE snapshot — synthesis-substitution UPDATEs at NVO 11167-11181 silently match 0 rows (or fail at EXEC with Msg 208 if the table is missing). Phase 6d closes this gap.
Implication for sub-phase 6d: UE must populate pfill_syn_powerfill_guide BEFORE re-invoking psp_powerfill_pool_guide per the legacy NVO 19795 ordering. After that re-invocation, the pool_guide rows for pa_key=1 get their price and prx_and_carry set from the synthesis. This is the legacy two-pass pattern PSSaaS preserves.
Needs confirmation from no one — empirical NVO + lifecycle observation.
A53: BR-3 Spec Enumerates Only 5 of 9 Legacy Pool-Action Values (VERIFIED 2026-04-19, SPEC AMENDED)
Claim: The PowerFill spec (powerfill-engine.md) §BR-3 currently enumerates 5 pool_action values: Remaining, Leaving, Joining, Switching, Swapped In (with Swapped In flagged as internal). The legacy psp_powerfill_pool_guide body emits 9 distinct pool_action values in its post-processing UPDATE blocks at NVO 11132-11181:
Remaining— derived from earlier INSERT blocks at NVO 10941 + 11062 (pool_action carried from##cte_posting_set_*upstream)Leaving— same upstreamJoining— same upstreamSwitching— same upstreamSwapped In— UPDATE at NVO 11157-11160:WHERE pool_action = 'Switching' AND pa_key = 1(the destination side of a Switch)Swapped Out— UPDATE at NVO 11162-11165:WHERE pa_key = 4 AND pool_action = 'Leaving'(the source side of a Switch)Vacated— UPDATE at NVO 11146-11149:WHERE pa_key = 5(loans whose source pool became empty after Switching)Ineligible— UPDATE at NVO 11151-11155:WHERE pa_key = 4 AND curr_status <> 'Eligible'(loans that would have left but aren't eligible)Eligible— UPDATE at NVO 11132-11134:SET curr_status = 'Eligible' WHERE curr_status = 'ELIGIBLE'(case normalization, not a pool_action change — but appears in the proc text)
Source: Direct NVO inspection (n_cst_powerfill.sru:11132-11181); empirical confirmation in 009 SQL output lines 3220-3253.
Confidence: High — direct NVO trace of the post-processing UPDATEs.
Disposition: (a) Corrected in place. Spec amended in this sub-phase to enumerate all 9 pool_action values with internal-state callouts:
- Remaining (user-facing): Loan was in pool X, stays in pool X
- Leaving (user-facing): Loan was in pool X, will be removed
- Joining (user-facing): Loan was unallocated, will be allocated to pool Y
- Switching (user-facing): Loan was in pool X, moves to pool Y. Generates paired internal
Swapped In(destination) +Swapped Out(source) - Swapped In (internal, pa_key=1): destination side of a Switch — used by Switching Thumbnail report (Phase 7)
- Swapped Out (internal, pa_key=4): source side of a Switch — paired with Swapped In
- Vacated (internal, pa_key=5): pool became empty after Switching
- Ineligible (internal, pa_key=5 — re-keyed from pa_key=4): loan would have left but isn't eligible
(Eligible is a curr_status normalization at NVO 11132-11134, not a pool_action value — clarified in spec amendment.)
Implication for Phase 7 read APIs: any API surface returning pool_action distribution (e.g., /runs/{run_id}/recap's pool_actions block) must enumerate all 9 values OR explicitly filter to user-facing 4 (Remaining/Leaving/Joining/Switching) per UI design intent.
Implication for Phase 7 Switching Thumbnail: filter on pool_action IN ('Swapped In', 'Swapped Out') to surface the paired Switch sides. (Spec previously only mentioned Swapped In.)
Needs confirmation from Tom/Greg: Optional. The 9 values are direct NVO trace; the user-vs-internal classification is interpretation that Tom/Greg could refine. Not blocking 6c.
A54: PS_DemoData Snapshot Triggers a Latent psp_powerfill_pool_guide PRIMARY KEY Violation on ##cte_posting_set_1300 (RESOLVED 2026-04-19 via ADR-021 §Narrow Bug-Fix Carve-Out)
RESOLUTION (2026-04-19): A54 is CLOSED via two surgical fixes inside psp_powerfill_pool_guide (009_CreatePoolGuideProcedure.sql), both within ADR-021's amended Narrow Bug-Fix Carve-Out scope:
- PK extension —
##cte_posting_set_1300extended from(trade_id, loan_id)(2 cols) to(trade_id, loan_id, pa_key)(3 cols), withpa_key NUMERIC(1, 0) NOT NULLadded to the table CREATE + the SELECT (using the identical CASE expressionLEFT(pool_action, 1) IN 'L'/'R'/'J'/elsethat_1400's SELECT already uses ~80 lines later). Mirrors_1400's author-intended shape byte-for-byte. pt13JOIN qualifier extension — addedAND ps1200_13.settlement_date = pt13.settlement_dateto the INNER JOIN clause. Thept13inline subquery already selected(trade_id, settlement_date)and GROUPed by both — but the legacy JOIN qualified ontrade_idalone, producing fan-out when a trade has multiple distinct settlement_dates across its loans. Empirical: trade36177868had 4 rows with 3 distinct settlement_dates inpfill_powerfill_guidemid-run — explosion frompt13produced 3 copies of the same(trade, loan, pool_action, pa_key)row, which the PK extension alone could not disambiguate. Adding thesettlement_datequalifier eliminates the fan-out at source.
Empirical-resolution arc (chronological):
| Run | Status | Step 5 (pool_guide) | Step 6 (ue) | Notes |
|---|---|---|---|---|
| Pre-fix runs (5 historical) | Failed | Dup-key (36177868, 3385000026) (2-col PK) | Not reached | A54 / A56 fail-fast cascade |
5859cd70 (post-PK-only-fix) | Failed | Dup-key (36177868, 3385000026, 3) (3-col PK; PK extension verified live; bug shape revealed as fan-out) | Not reached | Diagnostic-First moment: PK fix necessary but not sufficient |
43e8f148 (post-both-fixes) | Complete | 515 rows | Succeeded (12 log events) | End-to-end success; A54 + A56 closed |
7c9dfe50 (verification) | Complete | 515 rows | Succeeded (12 log events) | Reproducible; identical outcome |
Process-discipline observation: the kickoff hypothesis (multi-pa_key Switching for the same (trade, loan)) was wrong. The actual bug is the pt13 JOIN qualifier omission; pa_key doesn't disambiguate the fan-out's identical-pool_action duplicates. Diagnostic-First Rule worked exactly as designed — the mid-run probe of pfill_powerfill_guide revealed 3 distinct settlement_dates for trade 36177868, which contradicted the plan-time hypothesis and led to the second surgical fix. Without the empirical step, the PK-only fix would have been shipped and discovered as insufficient at the post-fix PoC anyway.
Banked for future kickoffs: when proposing carve-out fixes for legacy proc bugs, include a "verify the fan-out source empirically before sizing the fix" step in the planning gate. The PK extension and the JOIN qualifier extension are independent fixes; both are needed; planning should anticipate this kind of decomposition.
Sentinel bumped to phase-8-superset-ready-a54-fixed (Architect-chosen; documents the fix in /api/powerfill/status without re-deriving from commit log).
Tom/Greg consultation: PO-confirmed demo-as-consultation per ADR-021 §Narrow Bug-Fix Carve-Out clause (d). The Greg-demo-readiness completion report (powerfill-a54-fix-greg-demo-readiness) is the consultation material Greg reviews live.
Phase 9 hook: the parallel-validation harness should compare per-loan allocations between PSSaaS (post-A54-fix) and the Desktop App on a customer DB to confirm the fix doesn't introduce per-loan divergence. Both fixes are strictly more permissive than the pre-fix proc (every input the pre-fix proc accepted, post-fix also accepts; rows that previously caused PK violations now succeed). No row is lost; no row is added that wasn't always conceptually present in the legacy author's intended logic.
See also:
- A56 — RESOLVED 2026-04-19 (Step 5 fail-fast cascade no longer applies; UE runs end-to-end)
- A65 (NEW) — Broader observation: multi-pa_key Switching + settlement-date variance are two distinct triggers; PS_DemoData snapshot exposes both; PS608 customer DBs apparently expose neither
- A66 (NEW) — UE clears + rebuilds-empty user-facing tables on syn-trade-empty datasets (post-A54 surfaced behavior; not a bug; documented expected post-UE state)
- ADR-021 §Narrow Bug-Fix Carve-Out — the precedent-setting amendment
psp_powerfill_pool_guidebody (NVO 8770-11185 /009_CreatePoolGuideProcedure.sqllines 1977-2125) — the amended proc body
A54 (HISTORICAL — original 2026-04-19 entry preserved for the empirical-resolution arc)
Claim: Running psp_powerfill_pool_guide against PS_DemoData (after 6b's allocation produces 515 rows in pfill_powerfill_guide) fails at runtime with:
Msg 2627: Violation of PRIMARY KEY constraint 'PK__##cte_po__...'.
Cannot insert duplicate key in object 'dbo.##cte_posting_set_1300'.
The duplicate key value is (36177868, 3385000026).
The temp table is created at NVO 9889-9919 with PRIMARY KEY (trade_id, loan_id) (2 columns). The INSERT at NVO 9921 selects from ##cte_posting_set_1200 LEFT JOIN cash-grid + INNER JOIN to ##cte_detail_0110 — the upstream ##cte_posting_set_1200 includes a pa_key column AND can produce multiple rows for the same (trade_id, loan_id) when the loan has multiple pa_key actions on the same trade (e.g., pa_key=1 Joining AND pa_key=4 Leaving rows for a Switching loan that's tracked on both source and destination sides). The PK on ##cte_posting_set_1300 omits pa_key and therefore cannot accommodate this — it's a legacy proc design issue that surfaces only against data shapes with multi-pa_key loan/trade pairs.
Note: The very next CTE in the proc body (##cte_posting_set_1400 at NVO 10001-10033) has PRIMARY KEY (trade_id, loan_id, pa_key) — 3 columns. The author clearly knew the right key shape for 1400; the 2-column key on 1300 looks like an oversight that was masked in production data shapes that don't trigger it.
Source: Empirical PoC observation 2026-04-19 against PS_DemoData. SqlException 2627. Affected loan/trade pair: (loan_id=3385000026, trade_id=36177868). Verified pfill_powerfill_guide has only 1 row for this pair (no upstream dup); the dup is generated by the proc's own CTE pipeline.
Confidence: High — reproducible empirical SqlException + direct NVO PK definition inspection + post-PoC table state verification.
Disposition: (c) Deferred with justification.
- The 6c port is byte-equivalent verbatim per ADR-021. Our
009_CreatePoolGuideProcedure.sqlfaithfully transcribes NVO 9889-9919's PK definition. Phase 9 parallel-validation against the Desktop App on this same data would surface the same failure. - Modifying the PK to
(trade_id, loan_id, pa_key)would deviate from verbatim port — that's a Desktop App bug fix, not a PSSaaS port concern. Such a fix needs explicit PO escalation + an ADR amendment + Tom/Greg consultation, since the upstream##cte_posting_set_1200row de-duplication semantics may have been intended (i.e., Tom/Greg may know that the legacy proc was supposed to receive de-duped input from a pre-allocation step PSSaaS hasn't yet implemented). - PS_DemoData is the snapshot data we have. Other Desktop App customer DBs likely don't have this exact data shape (cleaner runs, fewer mid-pa_key collisions). Phase 9 validation against a real customer DB is the gate.
- The 6c success criterion is partially met: Step 5 IS reached, EXECs the proc for ~31s on real data, and the failure is informative (it surfaces the specific data shape triggering it). The pre-failure parts of the proc body executed correctly; just the
INSERT INTO ##cte_posting_set_1300at NVO 9921 hit the PK violation.
Implication for sub-phase 6c PoC: RunSummary.pool_guide_count = 0 on PS_DemoData (the proc body fails BEFORE the INSERT INTO dbo.pfill_pool_guide blocks at NVO 10941 / 11062 / inner-EXEC-insert4). The pool_action distribution cannot be exhibited until A54 is addressed — either by (a) PS_DemoData data cleanup, (b) Phase 9 testing against a different DB, or (c) Tom/Greg ADR-amendment escalation to fix the PK.
Implication for Phase 9 parallel-validation: the harness must include a "data-shape compatibility" pre-flight that detects multi-pa_key pairs in the upstream and either skips the PoC, normalizes the data, or surfaces the same Desktop App failure for parity.
Implication for Phase 6d (UE pass): UE re-invokes psp_powerfill_pool_guide at NVO 19795. UE will hit the same A54 failure on PS_DemoData unless something between 6c and 6d's UE invocation resolves the upstream multi-pa_key state. Phase 6d Architect should expect this and plan around it.
Family relationship:
- A49 (column-missing CREATE failure on local pssaas-db) — different layer: A49 is CREATE-time deferred name resolution limit. A54 is runtime PK violation. Both are "tenant-DB-specific failure modes."
- A51 (parallel candidate-build observation) — same family as "the conset proc has a more complete view of the data than 6a's C# candidate-builder." A54 says "the legacy pool_guide proc has a more demanding view of the data than the snapshot can satisfy."
- R-DATA-1 / R-DATA-2 family — PS_DemoData is a fixed snapshot; some legacy code paths require data shapes that the snapshot doesn't have. R-DATA-1 was 6a's (FHLMC small-balance instruments); R-DATA-2 is now 6c's (multi-pa_key loan/trade pairs).
Needs confirmation from Tom/Greg (deferred to Phase 9):
- Was the 2-column PK on
##cte_posting_set_1300intentional or an oversight? - If intentional, what upstream invariant guarantees no multi-pa_key duplicates? Has that invariant been violated in PS_DemoData by some sanitization step?
- If oversight, can the PK be fixed in PSSaaS (with ADR-021 amendment for "narrow legacy bug fixes")?
Needs confirmation from PO (non-blocking): acknowledgment that Phase 6c ships with the partial PoC outcome documented; Phase 9 is the remediation gate.
A55: PS_DemoData Empirical PK Probe Reveals 4-Table Key Shapes; Corrects Syn-Trades Deep Dive Column-Type Hypothesis (VERIFIED 2026-04-19)
Claim: A direct sys.indexes probe against PS_DemoData (2026-04-19) recovered the empirical primary-key configurations for the 3 pfill_syn_* tables that the syn-trades deep dive could only partially infer:
| Table | Empirical PK | Source |
|---|---|---|
pfill_syn_trade_base | (syn_trade_id, rate) CLUSTERED | sys.indexes |
pfill_syn_powerfill_guide | (loan_id) CLUSTERED | sys.indexes |
pfill_syn_powerfill_guide_all_rank | NO PK (heap) | sys.indexes confirms type=2 ROWSTORE without is_primary_key=1 |
pfill_powerfill_log | not present in PS_DemoData; 6d adopts (log_id IDENTITY) per NVO 13267-13274 auto-create block | NVO + PSSaaS deploy decision |
The syn-trades deep dive's column-type hypothesis is empirically wrong on one point (F-6d-3): the deep dive said pfill_syn_powerfill_guide.syn_trade_id was the only int column in the subsystem ("likely IDENTITY"). PS_DemoData reality:
pfill_syn_trade_base.syn_trade_id=int(10,0) IDENTITY(1,1)✓ matches deep dive's "likely IDENTITY" but on the wrong tablepfill_syn_powerfill_guide.syn_trade_id=varchar(30) NOT NULL(NOT a key on this table; the table's PK isloan_idalone)pfill_syn_powerfill_guide_all_rank.syn_trade_id=varchar(30) NOT NULL(no PK on this table)
Interpretation: syn_trade_id is a foreign-key-ish reference INTO pfill_syn_trade_base's int PK; it's stored as varchar(30) in the consumer tables to match other ID-column conventions in the schema (loan_id, trade_id, current_trade_id all varchar(30)). The deep dive author saw the int column and assumed it propagated; it doesn't.
Source: Direct PS_DemoData sys.indexes query 2026-04-19 (Phase 6d planning Primary-Source Verification Gate); empirical information_schema.columns cross-reference.
Confidence: High — direct catalog probe.
Disposition: (a) Corrected in place. The 6d schema script 010_CreatePowerFillSynTradesSchema.sql uses the empirical PK shapes verbatim. The 4 EF entities encode them via [Key] (single-col), IEntityTypeConfiguration<T>.HasKey() (composite), and HasNoKey() (heap). The syn-trades deep dive doc's column-type hypothesis is footnoted via this assumption rather than rewritten in-place (the deep dive's structural narrative is otherwise accurate; only the one column-type guess was wrong).
Implication for Phase 7 read APIs: any API surface paginating over pfill_syn_powerfill_guide_all_rank must use a stable ORDER BY (e.g., loan_id, payup_plus_carry_rank) since the table is keyless and rows have no inherent uniqueness ordering.
Implication for Phase 9 parallel-validation: the harness should compare per-loan rows in pfill_syn_powerfill_guide (PK on loan_id makes per-loan diff trivial); for _all_rank, comparison must be by (loan_id, syn_trade_id) composite even though it's not a PK.
Needs confirmation from no one — empirical fact + deploy verification.
A56: Phase 6d PoC Reveals A54 Step 5 Fail-Fast Short-Circuits Step 6/UE on PS_DemoData (RESOLVED 2026-04-19 with A54)
RESOLUTION (2026-04-19): A56 is CLOSED as a side-effect of A54's resolution. With A54's two surgical fixes (PK extension + pt13 JOIN qualifier extension), Step 5 (pool_guide) succeeds end-to-end on PS_DemoData; Step 6 (ue) is no longer short-circuited by the fail-fast contract; UE runs to completion (run 43e8f148-... and 7c9dfe50-...). UE's inner pool_guide re-EXEC at NVO 19795 also succeeds (the same A54 fix applies — same proc body). The "doubly-Phase-9-blocked" framing no longer applies.
However, A56's resolution surfaced a new finding — A66 — about UE's clear-and-rebuild-empty behavior on syn-trade-empty datasets like PS_DemoData. UE structurally runs end-to-end (no SqlException; sentinel + tests + pfill_powerfill_log 12 events), but rebuilds the user-facing run-output tables to 0 rows because PS_DemoData lacks the syn-trade-eligible loan/trade pairs UE's logic depends on. This is documented expected behavior, not a defect. Phase 9 parallel-validation against a customer DB with real syn-trade opportunities is the gate to confirm UE's full rebuild logic.
See A66 below for the UE clear-and-rebuild observation.
A56 (HISTORICAL — original 2026-04-19 entry preserved for the empirical-resolution arc)
Claim: The 6d end-to-end PoC against PS_DemoData hits A54 in Step 5 (pool_guide) at the IDENTICAL line/loan/trade as the 6c PoC (SqlException 2627: Violation of PRIMARY KEY constraint 'PK__##cte_po__...'. Cannot insert duplicate key in object 'dbo.##cte_posting_set_1300'. The duplicate key value is (36177868, 3385000026).). Per the PowerFillRunService fail-fast contract established in 6a-6c, Step 6 (ue) is never reached when Step 5 fails. Consequently:
- The 4 new tables (
pfill_syn_trade_base,pfill_syn_powerfill_guide_all_rank,pfill_syn_powerfill_guide,pfill_powerfill_log) are deployed (per Deploy Verification Gate arm c — OBJECT_ID returns non-null on both pssaas-db and PS_DemoData) but are not populated during the PoC. - The Step 6 UE proc body is structurally deployed (
OBJECT_ID('dbo.psp_powerfillue', 'P') IS NOT NULLon PS_DemoData) but is not exercised end-to-end. - The 6 new
RunSummaryfields (syn_trade_base_count,syn_powerfill_guide_all_rank_count,syn_powerfill_guide_count,pfill_powerfill_log_count,post_ue_allocated_count,post_ue_pool_guide_count) all return0in the PoC RunResponse.
The doubly-Phase-9-blocked nature: even if I bypassed Step 5's fail-fast (an architectural deviation that contradicts the 6c-established orchestration contract), UE itself re-invokes pool_guide internally at NVO 19795 (per F-6d-7 verification + the unmodified 011 verbatim port). The same PK violation would fire from inside UE, just at a different observable point.
Source: 6d PoC run 88a7f1ac-0813-433a-a8b3-c1f5706e2515, 2026-04-19 16:43:26 UTC. Full RunResponse JSON captured in the 6d completion report. PS_DemoData empirical post-PoC state: pfill_powerfill_guide 515 rows (matches 6b/6c baseline; Step 4 wrote it), all 4 new tables 0 rows, pfill_pool_guide 0 rows.
Confidence: High — empirical PoC + identical SqlException to 6c + unchanged data state confirmed via SELECT COUNT(*).
Disposition: (b) Scope-changed already accounted for. This is the documented Option C outcome that PO confirmed at the 6d planning checkpoint. The 6d port is structurally correct (deploys clean, OBJECT_IDs verifiable, build/test green); the runtime exercise is blocked by A54 (Phase 9 carry-over). The 6d Architect did NOT silently work around — Andon-cord was pulled, root cause confirmed, disposition matches 6c precedent.
Implication for Phase 9 parallel-validation harness:
- Necessary: a "data-shape compatibility" pre-flight that detects multi-pa_key duplicates in upstream tables and either skips PoC, normalizes, or surfaces the same Desktop App failure for parity (already noted in A54 disposition).
- Sufficient (for 6d UE specifically): a fixture (clean Desktop App customer DB or A54-fixed PS_DemoData) that lets Step 5 succeed enables Step 6 to attempt UE; UE's own NVO 19795 re-invocation of pool_guide must ALSO be considered against the same PK constraint. Two failure points to satisfy, not one.
Implication for the 6d completion report: the report explicitly documents which 6d artifacts ARE empirically validated (deploy + entity registration + sentinel + tests + Step 5 unchanged) vs which are STRUCTURALLY validated only (Step 6 / UE proc body / 6 new RunSummary fields). The completion report's PoC verification commands section makes this distinction explicit so the Collaborator + PO + a future Phase 9 Architect can recover the disposition without re-deriving it.
Needs confirmation from no one — empirical PoC outcome + matches predicted Option C.
A57: Phase 6d Kickoff Cited NVO Line Numbers at Granular Specificity; Zero Net-New Truth Rot Findings (OBSERVATION 2026-04-19, BANKED for Process Discipline v3.1)
Claim: The 6d kickoff cited 15+ specific NVO line numbers (NVO 13246, 13265-13275, 13441-13451, 16463, 16465, 19795, 19801, plus 5 others). The 6d Architect's 3-layer Primary-Source Verification Gate produced 0 net-new Truth Rot findings against the kickoff itself. F-6d-1, F-6d-2, F-6d-5, F-6d-6, F-6d-7 all confirmed kickoff claims as exact. The only finding that contradicted derived doc was F-6d-3 — a column-type hypothesis in the syn-trades deep dive (a different upstream artifact, also Architect-authored).
This is the first Phase 6 sub-phase where the kickoff was clean on its primary anchors. Pattern observation:
| Sub-phase | Kickoff specificity at NVO-line level | Net-new Truth Rot findings against kickoff |
|---|---|---|
| 6a | Mostly ranges ("NVO 7500-12500") | F-VERIFY-1, F-VERIFY-2 (2 findings) |
| pre-6b sweep | Range hint ("NVO 6500-6700") | F-Sweep-1 (1 finding: range was 6004-6756) |
| 6b | Few line citations ("NVO 7500-12500" reused) | A46 (kickoff hint wrong) |
| 6c | Some line citations ("NVO 7194-11186" for pool_guide) | A38, F-6c-2 (2-of-11 findings against kickoff) |
| 6d | 15+ specific line citations (13246, 13265-13275, 13441-13451, 16463, 16465, 19795, 19801, etc.) | 0 |
Interpretation: Granular NVO citations at the line-number level are self-verifying — the gate's NVO-vs-doc layer becomes a fast Boolean check (line N either matches the cited content or doesn't). Vague ranges or hand-waving ("the multi-pass blocks scattered through NVO 7500-12500") accumulate drift between when the doc was written and when the next sub-phase's Architect reads it.
Source: The 5 sub-phase kickoffs (6a/pre-6b/6b/6c/6d). All Architect-authored or Collaborator-authored. Pattern is observable across both author types.
Confidence: Medium-High for the pattern; High that 6d's specific kickoff was 0-finding. The pattern is correlation; specificity may be an outcome variable rather than the cause (e.g., later sub-phases may benefit from prior sub-phases having corrected the upstream artifacts the kickoffs cite).
Disposition: (b) BANKED OBSERVATION for Process Discipline v3.1 nomination. The candidate process refinement "Reference docs are not primary source — even Collaborator-authored kickoff documents AND Architect-authored sub-phase plans must be Primary-Source Verification Gate inputs themselves" (3-session-corroborated through 6c per PO acknowledgment) gets a complementary observation: kickoff specificity at NVO-line level is itself a Gate-quality signal. Future kickoff drafting practice could include a "line-citation audit" step that converts every range/hand-wave into a specific NVO line where possible.
Implication for Phase 6e kickoff drafting: the 6d Architect recommends the 6e kickoff (when written) follows the 6d kickoff's specificity pattern — every NVO citation is a single line number or a tight range with explicit start/end content anchors.
Needs confirmation from PO (non-blocking): whether to advance this observation to a v3.1 nomination alongside the existing 3-session-corroborated "Reference docs are not primary source" candidate.
Phase 6e corroboration (2026-04-19): the 6e prompt itself was the second consecutive sub-phase with 0 net-new Truth Rot findings against the kickoff. F-6e-1 through F-6e-8 verified spec/doc/implementation anchors; F-6e-1 corrected an enum gap in the implementation (RunStatus had only 2 values vs spec's 6+) — that's a derived-doc-vs-implementation finding, not a kickoff-vs-source finding. The 6e prompt is a structured derivative of the breakdown + open-questions docs (themselves Architect-authored); the breakdown's specificity at NVO-line-citation level + the open-questions' explicit Q-by-Q PO confirmations + the 6d completion report's clear Architect-recommendation-for-6e section combined to produce a kickoff with very few Boolean-verifiable claims that could be wrong. A57's pattern observation now has 2-session corroboration — worth the v3.1 nomination drafting at the next process-discipline revision.
A58: BR-9 Cleanup Scope — User-Facing Tables Cleared, Syn-Trades + Log Preserved (VERIFIED 2026-04-19)
Claim: Phase 6e's BR-9 failure-state cleanup (PowerFillRunHistoryService.CleanupRunOutputTablesAsync) clears the 7 user-facing run-output tables (pfill_loan2trade_candy_level_01, pfill_powerfill_guide, pfill_kickout_guide_01, pfill_trade_base, pfill_cblock_guide, pfill_trade_cblock_base, pfill_pool_guide) on Failed/Cancelled runs. The 4 syn-trades + log tables (pfill_syn_trade_base, pfill_syn_powerfill_guide_all_rank, pfill_syn_powerfill_guide, pfill_powerfill_log) are intentionally PRESERVED — they retain forensic value as evidence of what UE managed to produce before the failure point.
Source: Phase 6e Architect plan §3 D5 + §4 A4.2 Alternatives-First disposition (PO-confirmed via Q7 Option B at the planning checkpoint). Empirically verified 2026-04-19 against PS_DemoData (run 909d7f16-... Failed status; post-cleanup query confirms all 7 user-facing tables = 0 rows; the 4 syn-trades + log tables = 0 rows because Step 6 never reached them per A56 — but had Step 6 fired and partially populated them before A54 in the inner pool_guide EXEC at NVO 19795, those rows would survive the cleanup).
Rationale (per A4.2 Option B over Option A "clear all"): UE writes the syn-trades + log tables at NVO 19019-19790 BEFORE the inner pool_guide EXEC at NVO 19795 fires the documented A54 PK violation. If/when end-to-end UE runs against an A54-fixed dataset (Phase 9), the syn-trades + log content is the canonical evidence of UE's behavior — clearing them on cleanup would discard that evidence. The 6d completion report's D9 (best-effort post-failure counter hoist) explicitly designed for this forensic observability; A58's cleanup-scope split preserves it in 6e.
Confidence: High — empirical PoC + 6d D9 design intent.
Disposition: (a) Documented in spec. Spec amended in this sub-phase to enumerate the 7 cleared tables in §BR-9 and to note the 4 preserved tables in §"Phase 6e PSSaaS-explicit tables".
Implication for Phase 7 read APIs: when a failed run is queried via GET /runs/{run_id}, the deserialised response_json shows whatever the orchestrator captured at finalise time (including the syn-trades + log counters from PopulatePostUeCountersAsync's post-failure hoist). The actual table rows are also queryable via Phase 7's /runs/{run_id}/syn-trades (or whatever the canonical Phase 7 route is) for any failed run that reached UE.
Implication for Phase 9 parallel-validation: the harness can use the syn-trades + log preservation to compare UE's intermediate output between PSSaaS and the Desktop App, even when end-to-end runs fail — which makes A56's "doubly-Phase-9-blocked" carry-over less blocking than it appeared in 6d (we can validate UE's Phase D + Phase E behavior in isolation against pre-A54-failure UE state).
Needs confirmation from no one — empirical fact + matches PO-confirmed Q7 Option B + matches 6d D9 design intent.
A56 — Phase 6e carry-over update (2026-04-19)
A56 (Step 5 fail-fast cascade prevents Step 6/UE on PS_DemoData) carries over to Phase 6e unchanged in disposition: 6e's PoC against PS_DemoData reproduces the IDENTICAL A54 outcome at the IDENTICAL line/loan/trade ((36177868, 3385000026)). 6e's contribution is the orchestration-layer validation (async behaviour, audit row creation, BR-8 enforcement, BR-9 cleanup, GET endpoints, cancel) that the 6d-and-earlier completion reports flagged as needing async-runtime infrastructure to verify.
6e PoC observation: the orchestration layer is fully exercised against the predicted A56 outcome:
- POST /run returns 202 + run_id with the new
RunSubmissionResponseshape ✓ - Background worker dequeues + transitions audit row through Pending → PreProcessing → Allocating → PostProcessing → Failed ✓
- Steps 1-4 produce the same 515-allocation baseline (no regression vs 6b/6c/6d) ✓
- Step 5 fails with the IDENTICAL A54 SqlException ✓
- Step 6 not recorded (fail-fast contract preserved) ✓
- BR-9 cleanup clears the 7 user-facing tables (verified via post-PoC SELECT COUNT(*)) ✓
- Audit row
failure_step='pool_guide'+failure_messagepopulated with the SqlException 2627 ✓ - BR-8: second concurrent POST /run returns 409 with
RunConflictResponsebody containing the active run id+status ✓ response_json(full RunResponse snapshot) is 2308 bytes;GET /runs/{run_id}round-trips it to the canonical 6a-6d shape ✓
Phase 9 remains the gate for end-to-end UE PoC observation per the 6d / 6c precedent.
A59: Phase 7 Kickoff Cited NVO 6730-6753 As The View Definition Source — That Is The Deploy Block, Not The Body (OBSERVATION 2026-04-19)
Claim: The Phase 7 Architect kickoff cited "NVO 6730-6753" as the source for pfillv_existng_pool_disposition view definition. Direct verification against n_cst_powerfill.sru: NVO 6730-6753 is the deploy/registration block in of_create_views() (it just calls of_get_pfillv_existng_pool_disposition()); the actual view BODY function of_get_pfillv_existng_pool_disposition() lives at NVO 12485 (with the SQL string-built through NVO 12640+).
Source: Direct NVO inspection 2026-04-19 (Phase 7 Three-layer Primary-Source Verification Gate, F-7-1).
Confidence: High — empirical Grep pfillv_existng_pool_disposition against the NVO returns 5 hits; only the 12485 function body is the SQL-text source.
Disposition: (b) Scope-changed. Phase 7 does NOT need to author or modify the view: PSSaaS-side 002_CreatePowerFillViews.sql (Phase 2) already deployed it from the correct NVO 12485 source. The kickoff mis-cite was inconsequential for Phase 7's actual scope (read endpoint over the existing view), but breaks A57's "kickoff specificity reduces Truth Rot" pattern observation at the 3-session-corroboration mark.
Pattern impact on A57: A57 had 2-session corroboration (6d kickoff + 6e prompt = 0 net-new findings against the kickoff). A59 is the first net-new finding against the kickoff in the Phase 6e → Phase 7 arc (one mis-cite out of ~12 NVO/spec-line citations in the Phase 7 prompt — still a low rate). The pattern observation persists: specificity at the line-number level remains a Gate-quality signal, and the mis-cite was self-correcting (one Grep against the NVO surfaced the right line). The v3.1 nomination drafting can still proceed with the qualifier: "kickoff specificity reduces but does not eliminate Truth Rot probability; the Gate's NVO-vs-doc layer remains a load-bearing check even with high-specificity kickoffs."
Implication for future kickoff drafting: When citing NVO line numbers as authoritative source for a function's BODY, verify the line range is the function definition itself (search for function string of_get_* or public function) rather than its deploy/invoker site. The Phase 7 mis-cite was understandable — both 6730 and 12485 are about the view — but the body content is at 12485.
Needs confirmation from no one — empirical NVO + lifecycle observation.
A60: Phase 7 {run_id} URL Parameter Semantics — Latest-Complete-Wins (DECIDED 2026-04-19, ADR-025)
Claim: Phase 7 endpoints take a {run_id} URL parameter, but the underlying pfill_* run-output tables hold ONE run's output at a time (BR-9 clears on Failed/Cancelled per A58; BR-10 overwrites on next Complete run). The URL parameter therefore needs a freshness verdict applied: the audit row in pfill_run_history is checked, the latest run for the tenant is identified, and the response shape varies by whether the requested run IS the latest:
- Latest + Active OR Complete → Current freshness; service queries the
pfill_*tables and returns the data; HTTP 200. - Latest + Failed/Cancelled → TerminalEmpty freshness; BR-9 cleared the user-facing tables, so empty payload + Note carrying failure_step + failure_message; HTTP 200 (because the data IS the empty state — telling the client "there is nothing to read for this run by design"). Reports reading non-BR-9-cleared sources (Existing Disposition, Cash Trade Slotting per A58) may still surface real data alongside the freshness Note.
- Older than the latest → Stale freshness; the requested run's data was overwritten by a later run; HTTP 410 Gone with the latest run id in the body Note.
- Doesn't exist for tenant → RunNotFound; HTTP 404 Not Found.
Source: Phase 7 Architect plan §3 A7.1 Alternatives-First decision (Option B chosen over Option A "strict per-run / 410 for everything not latest" and Option C "snapshot replay" deferred per spec line 263). Documented in ADR-025.
Confidence: High — explicit Architect decision with Alternatives-First framing; ADR-025 records full provenance.
Disposition: (a) Documented in ADR-025 + spec amendment. The spec's §Output APIs section is rewritten in Phase 7 to enumerate the freshness verdicts + their HTTP status codes + their Note conventions.
Implication for Phase 8 (UI): the Note field is the load-bearing UX signal. Clients should branch on run_status and note substring match to render appropriate freshness indicators (e.g. "this report shows historical data from run X, but a newer run Y has run since" → "view latest run" button bound to the latest run id from the Stale Note).
Implication for Phase 9 (Parallel validation): the harness can compare PSSaaS reports to Desktop App reports per-loan / per-trade. The Cash Trade Slotting + Existing Disposition reports (which read non-BR-9-cleared sources) provide validation paths even when A54 short-circuits Step 5 on PS_DemoData (per F-7-8 demonstration: 688 cash_market_map rows surface from a Failed run).
Needs confirmation from no one — Architect decision; PO-acceptance via the Phase 7 completion review.
A61: PSSaaS-Side EF Entity for pscat_trade_cash_grid (ADDED 2026-04-19)
Claim: Phase 7's Cash Trade Slotting report's source-of-truth join target is pscat_trade_cash_grid (per spec line 442 + the legacy plugin's BX cash-grid output from Step 1 psp_pfill_bx_cash_grids). PSSaaS-side EF entity for this table did not exist pre-Phase 7 (the procedures in 003+006+008+011 reference it as a SQL-side table without a PSSaaS-side entity wrapper). Phase 7 adds it as a PowerFill-owned Upstream entity at src/backend/PowerSeller.SaaS.Modules.PowerFill/Domain/Upstream/PscatTradeCashGrid.cs with composite PK (trade_id, rate, syntax_name) matching infra/sql/init/seed-schema.sql lines 449-456.
Source: Phase 7 plan F-7-5 finding + the seed-schema DDL.
Confidence: High — DDL transcription + matches legacy customer-DB column shapes.
Disposition: (b) Scope-changed — added in Phase 7. The entity is type-safe but is NOT actually JOINed by the Phase 7 service yet; the Cash Trade Slotting report surfaces pfill_cash_market_map rows with null trade_id / sequence_number / price / trade_amount fields (per A12: PS_DemoData runs skip Step 1 because bx_price_floor is null). Phase 9 closes this when a customer DB exercises the report end-to-end (the JOIN can then be added to the service without breaking the contract).
Implication for EntityConfigurationTests.cs: the test scopes to Domain namespace entities; PscatTradeCashGrid lives in Domain.Upstream and is not picked up by the test (matches the existing pattern for other upstream entities like Trade, Pool, etc.). No test update needed.
Needs confirmation from no one — empirical DDL + entity scaffolding.
A62: PS_DemoData Has The Legacy WITH ENCRYPTION Version of pfillv_existng_pool_disposition View (Pre-Dates note_rate Column) (VERIFIED 2026-04-19)
Claim: The pfillv_existng_pool_disposition view on PS_DemoData throws SqlException 207: Invalid column name 'note_rate' when queried via PSSaaS's PoolDispositionReadModel. The PSSaaS-side 002_CreatePowerFillViews.sql (Phase 2) defines the view with a note_rate column (transcribed from NVO 12485 + the corresponding deep dive); the legacy view on PS_DemoData is the older WITH ENCRYPTION version that pre-dates the note_rate addition. Per Backlog #24 (also referenced in A30 RESOLVED disposition), the PS_DemoData view deploy is DEFERRED pending a behavior diff against the encrypted definition.
Source: Empirical Phase 7 PoC against PS_DemoData run 769245cf-d317-411f-87aa-1d4345fb8489 (2026-04-19). Initial GET against /runs/{run_id}/existing-disposition returned Microsoft.Data.SqlClient.SqlException (0x80131904): Invalid column name 'note_rate'. Phase 7 service updated to catch SqlException 207/208 and degrade to an empty payload + explanatory Note.
Confidence: High — reproducible runtime SqlException + matches A30's documented Backlog #24 deferral.
Disposition: (a) Corrected in place — graceful degradation. Phase 7's GetExistingDispositionAsync catches SqlException 207/208 and returns an empty payload with a Note that explicitly references A62 + Backlog #24. Clients see the issue rather than a 500.
Implication for Phase 9 parallel-validation: the harness should detect this schema-drift early and either (a) deploy 002_CreatePowerFillViews.sql to PS_DemoData (now possible per A30 RESOLVED — kevin_pssaas_dev has DDL perms) and accept the behavior-diff against the encrypted definition, OR (b) rename the PSSaaS view to pfillv2_* to avoid the collision. The current PSSaaS-side definition matches the post-note_rate legacy version; the PS_DemoData view is older. Decision deferred per Backlog #24 disposition.
Implication for Phase 7 PoC verification: the existing-disposition endpoint returns 200 OK + empty payload + dual-Note (TerminalEmpty + schema-drift). The endpoint's 8/8 PoC verification therefore demonstrates: (i) the freshness verdict mechanism, (ii) the catch-and-degrade resilience pattern, (iii) the Note composition convention (semicolon-delimited multi-source explanation).
Needs confirmation from PO (non-blocking): acknowledgment that Phase 7 ships with this graceful-degradation behavior; Phase 9 closes the underlying view deploy decision.
Needs confirmation from Tom/Greg: is the legacy view's column shape (without note_rate) the canonical Desktop App version, OR is note_rate a recent addition Tom shipped that didn't propagate to PS_DemoData's snapshot? If the latter, the PSSaaS-side view + PS_DemoData behavior diff at Phase 9 should be straightforward.
A63: Phase 8 Existing-Disposition Dashboard Bypasses the Encrypted PS_DemoData View (DEFENSIVE QUERY) (DECIDED 2026-04-19)
Claim: Phase 8 Workstream 1's 42_pfill_existing_disposition.sql query intentionally does NOT reference dbo.pfillv_existng_pool_disposition. PS_DemoData has the legacy WITH ENCRYPTION version of the view that lacks the note_rate column (per A62); the Phase 7 PowerFillReportService catches the resulting SqlException 207 and gracefully degrades, but Superset's chart pipeline cannot. The Phase 8 SQL therefore mirrors the view's logical shape (CTEs cte_epd_0030 through cte_epd_0600 from 002_CreatePowerFillViews.sql) directly against the upstream tables (loan + rmcat_loan + loan_shipped + pfill_pool_guide) AND OMITS the note_rate column entirely (the column the legacy encrypted view also lacks).
Source: Phase 8 plan §2 Three-layer Primary-Source Verification Gate finding F-8-BR-1 (caught by the Backlog re-read pass per Phase 7 CR #1 — Backlog row #24's "encrypted view" disposition was the smoking gun before the live PoC surfaced anything).
Confidence: High — the defensive query was deployed to PSX Superset and verified end-to-end through Superset's engine on 2026-04-19; it executes cleanly against PS_DemoData (no SqlException 207 — F-8-BR-1 mitigation works) and returns 0 rows because the at-risk join requires rmcat_loan.segment_name = 'at risk' rows AND the pfill_pool_guide LEFT JOIN can't enrich on a BR-9-cleared table on the latest Failed run. Both empty conditions are PS_DemoData-snapshot artifacts, not query bugs.
Disposition: (b) Scope-changed — defensive query in 42_pfill_existing_disposition.sql + dashboard 6 reads it directly. Phase 9 closes either by (a) deploying 002_CreatePowerFillViews.sql to PS_DemoData (now possible per A30 RESOLVED) and reverting Dashboard 6 to query the view, OR (b) renaming the PSSaaS-side view to pfillv2_* to avoid the collision and exposing the legacy encrypted view alongside it. Until Phase 9 closes, the defensive query is the canonical Phase 8 W1 source for this dashboard.
Banked process observation (qualifies Phase 7 CR #1's nomination): the Backlog re-read pass at planning time DID catch this finding before the PoC surfaced it — first empirical demonstration of the candidate practice's value. Phase 7's F-7-7 was Backlog-discoverable but only surfaced at PoC; Phase 8's F-8-BR-1 was Backlog-discovered at planning. The candidate practice is now proven; Phase 8 completion report writes it up for canonical-promotion review.
Implication for Phase 9 parallel-validation: when Phase 9 closes A62 (deploy 002 to PS_DemoData OR rename PSSaaS view), Dashboard 6 + the Phase 7 service should both revert to the view (single source of truth). Until then both code paths (Phase 7 service catch-and-degrade; Phase 8 dashboard defensive-query) are independently valid; the Phase 9 closer must update both.
Needs confirmation from no one — Architect decision; PO acknowledgment via Phase 8 W1 completion review.
A64: Phase 8 v1 Superset Connection is PS_DemoData-Only — Multi-Tenant Registration Deferred to Phase 9+ (DECIDED 2026-04-19)
Claim: Phase 8 Workstream 1's Superset deployment uses a single existing PS_DemoData database connection (registered in PSX Superset as id=2 / name="PS_DemoData (PSSaaS Demo)" per setup-guide.md). Per-tenant Superset registration (one connection per tenant DB; or one Superset workspace per tenant via Superset RBAC) is explicitly deferred to Phase 9+ per the Phase 8 kickoff §Explicit scope (OUT) ("Auth / RBAC / multi-tenant isolation hardening — Phase 9+").
Source: Phase 8 plan §3 Alternatives-First Gate decision D-8-3 (chose Option C "PS_DemoData only for v1; multi-tenant Superset registration is Phase 9+ work"). Matches the existing 36-query infrastructure reality (all of infra/superset/queries/01_*.sql through 36_*.sql point at the same single PS_DemoData connection).
Confidence: High — explicit Architect decision with Alternatives-First framing; mirrors existing 5-script-per-domain pattern for BestEx, Position Recon, Pipeline, Trading, Loan P&L — every existing dashboard is PS_DemoData-only.
Disposition: (a) Documented in this assumptions log + the PowerFill Dashboards doc. The deploy script's DB_ID = 2 constant is the single per-environment configuration point; production / per-customer rollouts replace it with the appropriate tenant DB connection ID.
Implication for Phase 9 / production rollout: when PSSaaS onboards a real customer (e.g. WTPO on PS608), the deploy script's DB_ID constant becomes a per-environment variable AND PSX infra registers a new Superset DB connection for the customer DB. Dashboard SQL stays unchanged (the pfill_* table names are identical across tenants per ADR-005 / ADR-006). The deploy script's idempotent UPDATE-or-CREATE behaviour means re-deploying against the new DB_ID produces a parallel set of dashboards (different database_id foreign keys on each SqlaTable row) without colliding with the PS_DemoData set.
Implication for Phase 9 multi-tenant validation: if Phase 9 needs to validate PowerFill output against multiple customer DBs simultaneously, the script can be parameterized (DB_ID as a CLI arg or env var) and run once per tenant; OR Superset RBAC can scope a workspace per tenant. Phase 9 chooses based on operator workflow needs.
2026-04-19 platform-tailwind note (UPDATED 2026-04-19 — migration COMPLETE): the PSX-Infra-driven migration of Superset from psx-staging to pss-platform (PO direction; PSX Infra-executed; ~3 min cutover) is COMPLETE. Hostname bi.staging.powerseller.com unchanged; all 20 dashboards / 56 charts / 77 datasets preserved; PowerFill dashboards 13-20 still resolve correctly. Multi-tenant registration story is now architecturally cleaner: platform-Superset has its own dedicated superset-db metadata pod (decoupled from PSX postgres backups) and per-product DB connections registered at the platform layer. DB_ID = 2 is now a per-product config, not per-deploy-environment. Deferral to Phase 9+ stands; ergonomics achieved.
Needs confirmation from no one — Architect decision; PO acknowledgment via Phase 8 W1 completion review. Phase 9 may revisit if the operator workflow surfaces a real multi-tenant Superset requirement.
A65: Multi-pa_key Switching + Per-Loan Settlement-Date Variance Are Two Distinct Triggers For The A54 PK Violation; PS_DemoData Snapshot Exposes Both, PS608 Customer DBs Apparently Expose Neither (BANKED OBSERVATION 2026-04-19)
Claim: A54's failure surface decomposes into two independent latent triggers that both fire on PS_DemoData but apparently never fire (or never co-fire) on the PS608 customer DBs running the legacy Desktop App proc body unmodified:
- Multi-pa_key for the same (trade, loan) pair — the original kickoff hypothesis.
_1300's 2-col PK can't accommodate Switching loans that produce paired Joining + Leaving rows for the same (trade, loan). Fix: extend_1300PK to 3 cols (mirrors_1400's author-intended shape). - Per-loan settlement-date variance within a trade — the actual fan-out source observed empirically in Diagnostic-First probing. The
pt13INNER JOIN qualifies ontrade_idonly despite the inline subquery selecting(trade_id, settlement_date)and GROUPing by both. When loans on the same trade have different settlement_dates inpfill_powerfill_guide, the JOIN explodes each_1200row N times — N copies of the same(trade, loan, pool_action, pa_key)tuple — which the PK extension alone cannot disambiguate. Fix: extend the JOIN qualifier withAND ps1200_13.settlement_date = pt13.settlement_date(mirrors author intent — the column was selected forpt13with no other downstream use).
Source: Empirical observation 2026-04-19 during the A54 fix PoC. Run 5859cd70-... (post-PK-only-fix) shifted the dup-key from 2-tuple to 3-tuple (36177868, 3385000026, 3) — same loan, all 3 dups had pa_key=3 (Remaining), proving the multi-pa_key hypothesis was wrong for this trigger. Mid-run probe of pfill_powerfill_guide against trade 36177868 confirmed: 4 rows / 3 distinct settlement_dates → pt13 produces 3 rows for that trade → 3-way fan-out → 3 identical (trade, loan, pa_key=3) copies into _1300.
Confidence: High — empirical 5-run dup-key signature evolution (2-tuple → 3-tuple → resolved) + mid-run probe + post-fix Complete run.
Disposition: (b) Banked observation — informs Phase 9 parallel-validation harness design. Phase 9's harness should include a "data-shape compatibility" pre-flight that probes both triggers on the target tenant DB:
- Multi-pa_key probe:
SELECT trade_id, loan_id, COUNT(DISTINCT pool_action) FROM pfill_powerfill_guide_during_pool_guide_step GROUP BY trade_id, loan_id HAVING COUNT(DISTINCT pool_action) > 1 - Settlement-date variance probe:
SELECT trade_id, COUNT(DISTINCT settlement_date) FROM pfill_powerfill_guide GROUP BY trade_id HAVING COUNT(DISTINCT settlement_date) > 1
If either probe surfaces non-empty results on a customer DB, the legacy Desktop App proc body (unmodified) would also fail at this point. The PSSaaS-fixed proc body succeeds. Phase 9 thus has a cleaner empirical comparison point: PS_DemoData is a known-trigger dataset; PS608 customer DBs may or may not be triggers — and PSSaaS's behavior is now testable against either (legacy fails on triggers, PSSaaS succeeds on both).
Open question for Greg/Tom (deferred to Phase 9 or earlier consultation): what data-shape invariants did the original Desktop App proc author assume? Was the 2-col PK + the trade-id-only pt13 JOIN intentional under an implicit guarantee that:
- (a) Per-loan settlement_dates within a trade always equal
pscat_trades.settlement_date(no per-loan variance), AND - (b) No (trade, loan) pair ever has multiple
pool_actionvalues
If both invariants hold for typical customer data, the legacy proc body works correctly without our fixes. PS_DemoData's snapshot violates at least invariant (a). PS608's data behavior is unknown — Phase 9 will tell us empirically.
Needs confirmation from no one to ship A54 fix. Greg/Tom may revisit Phase 9 to confirm the broader pattern.
A66: psp_powerfillUE Clears + Rebuilds-Empty The User-Facing Run-Output Tables On Syn-Trade-Empty Datasets Like PS_DemoData (NEW 2026-04-19, surfaced by A54/A56 resolution)
Claim: With A54 + A56 resolved (Step 5 + Step 6 now run end-to-end), the PoC against PS_DemoData reveals a new behavior that was previously masked by A56's fail-fast cascade: psp_powerfillUE clears and rebuilds the user-facing run-output tables based on syn-trade-augmented logic. On PS_DemoData (which has no syn-trade-eligible loan/trade pairs satisfying UE's @pfill_map predicates), the rebuild produces 0 rows in:
pfill_powerfill_guide(UE-rebuilt, was 515 mid-run after Step 4)pfill_pool_guide(UE-re-EXEC'd pool_guide writes 0 rows)pfill_trade_base,pfill_loan2trade_candy_level_01,pfill_kickout_guide_01,pfill_cblock_guide,pfill_trade_cblock_base(UE-cleared)pfill_cash_market_map(UE DELETEs at NVO equivalent of011_*.sql:270-271then INSERTs 0 rows from empty@pfill_map→ ends at 0)
Empirical: post-A54-fix run 7c9dfe50-1b7d-471f-832a-d55053d2e7c5 summary reports allocated_count: 515, pool_guide_count: 515, post_ue_allocated_count: 0, post_ue_pool_guide_count: 0. pfill_powerfill_log populated with 12 events tracing UE's clear-and-rebuild sequence (events 1-3: clear+repopulate-empty Step-4 output; events 4-12: clear+attempt-syn-trade-synthesis-empty + final pool_guide re-EXEC). All 11 pfill_* data tables (excluding pfill_run_history audit) end at 0 rows post-Complete; pfill_powerfill_log retains 12 forensic events.
Source: Empirical PoC observation 2026-04-19 against PS_DemoData. UE proc body (011_CreatePowerFillUeProcedure.sql, ported verbatim from NVO 13246-19801) DELETE-INSERT pattern verified against the 4 data tables UE touches directly + the inner pool_guide re-EXEC at NVO 19795 propagating the empty rebuild downstream.
Confidence: High — empirical PoC + log-event trace + table-count verification + NVO source verification.
Disposition: (b) Documented expected behavior — not a bug. This is the legacy psp_powerfillUE's designed semantics on a dataset without arbitrage opportunities. The UE pass is the post-allocation pass; on customer DBs with syn-trade-eligible loan/trade pairs, UE rebuilds the user-facing tables with the synthetic-trade-augmented allocation. PS_DemoData's snapshot apparently doesn't have those pairs (the loan profile probably doesn't satisfy UE's strict eligibility predicates — see UE NVO 13800-15000 candidate logic for the empirical preconditions to investigate).
Implication for the Greg-demo narrative (per ADR-021 §Narrow Bug-Fix Carve-Out demo-as-consultation disposition): the post-A54-fix PoC demonstrates end-to-end Complete status but the user-facing dashboards 14-19 still render 0 rows post-UE. The Greg-demo narrative should emphasize:
- End-to-end Complete is now empirically achievable (was impossible pre-fix; ~30s wall-clock; the prior 5 Failed runs are visible in the Hub dashboard for direct comparison)
- The "Bug as Feature" arc gets stronger — PSSaaS surfaced two distinct latent legacy bugs (PK shape + JOIN qualifier omission), shipped both surgical fixes within ADR-021's amended carve-out, and now the proc body completes the full pipeline
- UE's clear-and-rebuild-empty behavior is a separate domain question — Greg can interpret the empty user-facing tables as a legitimate domain-correct outcome ("UE didn't find any synthetic-trade opportunities in this dataset") rather than a fix bug. This becomes a new Greg-consultation point: are PS_DemoData's loan profiles realistic representatives of typical customer data, or is the snapshot atypically syn-trade-poor?
- Dashboard 1 (Hub) is the new canonical proof-of-life — shows 11 runs total, latest is
Completewithoutput_guide_count=515. Dashboard 8 (Cash Trade Slotting) which previously showed 688 rows now shows 0 (UE supersedes Step 4'spfill_cash_market_mapper the same A66 mechanism — sub-finding of A66, see also A58 amendment below).
Implication for A58 (BR-9 cleanup scope split): A58 asserted that the 4 syn-trades + log tables are "preserved across BR-9 cleanup" because UE writes them BEFORE the inner pool_guide EXEC fired the A54 PK violation. That preservation was a side-effect of A56's fail-fast — UE never got past its early INSERTs. With A54+A56 resolved, UE runs to completion and may now clear those tables itself as part of its rebuild semantics. A58's "preservation" framing should be re-read with A66 in mind: the syn-trades + log tables are preserved across BR-9 (post-Failed-run cleanup), but UE's normal end-to-end run rebuilds them per its own logic. On PS_DemoData (no syn-trades), UE writes 0 rows; the only forensic surface that survives is pfill_powerfill_log (12 events for run 7c9dfe50-...). On customer DBs with real syn-trades, UE will populate them with the synthetic-trade-augmented allocation per the legacy author's design intent.
Implication for Phase 9 parallel-validation: the harness should add a "data-shape syn-trade probe" that detects whether the target tenant DB has loan/trade pairs satisfying UE's syn-trade synthesis predicates BEFORE running an end-to-end Complete-run comparison. On a syn-trade-empty dataset (like PS_DemoData), per-loan parity between PSSaaS and the Desktop App should be on the mid-run state (post-Step-4 pfill_powerfill_guide 515 rows, pre-UE-clear). On a syn-trade-rich dataset, parity should be on the post-UE state.
Needs confirmation from Greg (NEW Phase 9 / Greg-demo carry-over): is the empty post-UE state the correct domain outcome on a syn-trade-empty dataset, OR is there an upstream PS_DemoData data-shape issue that should populate syn-trades but isn't? If the latter, A66's resolution path is "fix PS_DemoData", not "fix UE".
Family relationship:
- A54 + A56 — A66 was masked by A54/A56's fail-fast cascade; A66 is the A54-resolution-surfaced-finding that the kickoff anticipated (per the "latent bugs masked by A54 firing first" warning).
- A58 — A58's "preservation across BR-9" framing needs re-reading with A66 in mind; UE itself supersedes the syn-trade-empty preservation when run end-to-end.
- A12 — A12 deferred Step 1 (
bx_cash_grids) on PS_DemoData; that defer is upstream of UE's syn-trade synthesis. Phase 9 should investigate whether running Step 1 with proper inputs would populate the upstream tables UE depends on for syn-trade synthesis.
A67: ReportContracts.cs XML Doc Comments Document a /reports/<name> Path Segment That Doesn't Exist in Routes (CODE-COMMENT TRUTH ROT) (DECIDED 2026-04-19, Phase 8 W2)
Claim: src/backend/PowerSeller.SaaS.Modules.PowerFill/Contracts/ReportContracts.cs lines 9-16 (the file-level XML doc comment) document the 8 Phase 7 endpoint paths as GET /api/powerfill/runs/{run_id}/reports/<report-name> (with a /reports/ path segment between {run_id} and the report name). The actual route registrations in src/backend/PowerSeller.SaaS.Modules.PowerFill/Endpoints/RunEndpoints.cs lines 331-451 use /runs/{runId}/<report-name> directly (no /reports/ segment). The Phase 7 + Phase 8 W1 + W2 kickoff documents all use the correct path; the discrepancy is confined to the in-code XML doc.
Likely origin: the /reports/ path segment was the kickoff's working assumption that got struck before route registration, but the contract-file XML doc-comment was committed against the earlier path shape and never re-aligned. The RunEndpoints.cs XML doc (line 30) ALSO shows /api/powerfill/runs/{run_id}/<report> (correct) — only ReportContracts.cs is stale.
Source: Empirical primary-source verification at Phase 8 W2 session start. Grep of MapGet|MapPost in RunEndpoints.cs returned 12 endpoint registrations matching the spec's documented paths; the contract-file XML doc paths do not match. Caught at Phase 8 W2 plan §2 Three-layer Primary-Source Verification Gate (Layer 1: Spec-vs-implementation), filed as F-W2-CONTRACT-1.
Confidence: High — empirical route enumeration vs static doc comparison.
Disposition: (a) Code against the actual RunEndpoints.cs routes in the Phase 8 W2 React UI (the canonical wire-shape source). The ReportContracts.cs XML doc-comment fix is deferred as cosmetic — fixing now would touch Phase 7's contract file outside W2's scope; the Phase 9 close-out (when the parallel-validation harness will exercise the contracts more aggressively) is the natural fix point.
Banking pattern: code XML doc comments are derived artifacts subject to Truth Rot exactly like spec/handoff documents. The Phase 8 W2 §2 Backlog re-read pass demonstrated this is now the 3rd corroborating instance of the candidate "Backlog re-read pass / re-verify against primary source at planning time" practice (Phase 7 F-7-7 anticipated; Phase 8 W1 F-8-BR-1 caught at planning; Phase 8 W2 F-W2-CONTRACT-1 caught at planning). The 3-instance corroboration may justify promoting "primary-source verification of code XML docs" to its own canonical sub-practice of the Implementation-vs-runtime layer at the next process-discipline revision.
Implication for Phase 9 parallel-validation: when the harness compares PSSaaS API responses to Desktop App outputs, the route paths it hits are the canonical contract — not the in-code XML doc. The harness can serve as the forcing function to align the doc comments at that time.
Implication for any future React/JS/external-consumer of the Phase 7 contracts: trust RunEndpoints.cs MapGet/MapPost registrations + the /api/powerfill/swagger/ OpenAPI document over any XML doc-comment in the contract files.
Needs confirmation from no one — Architect decision to defer the XML doc fix; PO acknowledgment via Phase 8 W2 completion review.
A68: tenant_id Column Value Is Conflated With "Which Connection-String Config Slot Was Used" Rather Than Logical Tenant Identity (PARTIALLY RESOLVED 2026-04-20 at Phase 8.5 W4: code shape decoupled; canonical-identity convention deferred to Phase 10+) (NEW 2026-04-19, surfaced by Phase 8 W2 staging deploy)
2026-04-20 status update (Phase 8.5 W4): PARTIALLY RESOLVED. Code shape decoupled per ADR-029:
- New
Tenantsealed record atsrc/backend/PowerSeller.SaaS.Infrastructure/Data/Tenant.cscarrying(TenantId, ConnectionString)as two independent fields TenantRegistry.Resolve(identity)returns theTenanttuple; oldGetConnectionString(tenantId)preserved for backward compatTenantMiddlewareresolution-precedence INVERTED: OIDCtenant_idclaim fromX-Forwarded-Access-Token(oauth2-proxy forward header from Phase 8.5 W1) wins over the legacyX-Tenant-Idheader- 9 new TenantMiddleware tests in
src/backend/tests/PowerSeller.SaaS.Api.Tests/Middleware/TenantMiddlewareTests.cspin: happy-path claim resolution; missing-claim default fallback; unknown-identity 401; claim-vs-header conflict (claim wins); backward-compat header-only; token-without-claim falls back to header; malformed-token falls back to header; helper-direct claim extraction (positive + missing-claim cases) - Frontend tenant-picker DISABLED with tooltip in
src/frontend/src/App.tsx(full removal deferred)
What's still deferred (per PO disposition Phase 8.5 plan §3 (d), 2026-04-20):
- The canonical-identity convention (UUID vs slug like
watermark-tpovssandboxvs hashed) is DEFERRED to first-real-customer-onboarding (Phase 10+). v1 keeps'ps-demodata'literally — both as the OIDC claim value (Keycloak realm-side mapper emitstenant_id: ps-demodatafor the demo user per cross-project-relay 2026-04-20 request item #2) AND as the persisted row tag value - The one-shot UPDATE migration script re-tagging historical
pfill_run_history.tenant_id='ps-demodata'rows is also deferred (no-op while the convention is unchanged)
Risks of the deferral (acknowledged + accepted): per A68 generalization shape (next-second-writer surprise), the next time a writer arrives needing to write to PS_DemoData (e.g., the Phase 9 harness post-OIDC-token-mint), the writer must use 'ps-demodata' for tenant_id — same constraint Path γ enforced on the staging API config slot, but now exposed at the Keycloak realm-mapper config rather than at K8s Deployment env vars. Migration cost grows as row count grows (currently ~12 rows; Phase 9 harness writes will add ~N per run).
ADR-029 documents the full long-term shape, the rationale for code-shape-only-v1, and the Phase 10+ follow-up checklist. The Phase 10+ canonical-identity-convention decision is a deliberate "decide with a real customer's organizational shape on the table" move.
Original A68 text follows for historical reference:
Claim: Throughout PSSaaS today (Phases 0-8), the tenant_id value written into multi-tenant rows (e.g. pfill_run_history.tenant_id) is sourced from _tenant.TenantId, which is set by TenantMiddleware from the inbound X-Tenant-Id header and used as both (a) the lookup key into TenantRegistry (selecting which connection string to use) AND (b) the stable customer-organization identity persisted on every row. These are logically distinct concerns conflated into one string.
In a single-writer PoC this never matters because there is only one route of writes per physical database, so tenant_id row-values are uniformly whatever-string-that-route-uses. PowerFill Phases 0-8 had only the Architect's local route writing into PS_DemoData, with TenantMiddleware resolving X-Tenant-Id: ps-demodata against the Tenants:ps-demodata config slot, so all 12 historical pfill_run_history rows got tagged tenant_id='ps-demodata'.
The conflation became load-bearing on the Phase 8 W2 staging deploy. Staging's API has its SQL MI connection-string mapped to the Tenants:default config slot (because that's the convention the staging Kubernetes manifests use), so a default-header request from the React UI would write rows tagged tenant_id='default' — invisible to local queries tagged 'ps-demodata'. Same physical database, two different tenant_id values, mutually invisible row sets.
Source: Empirical Phase 8 W2 staging deploy verification post d4d294e. Direct DB query SELECT tenant_id, COUNT(*) FROM pfill_run_history GROUP BY 1 against PS_DemoData via the SQL MI public endpoint returned 12 rows ALL tagged tenant_id='ps-demodata'; staging API queries WHERE tenant_id='default' returned 0 rows, identical schema, identical underlying data.
Confidence: High — empirical primary-source verification at the database row level. The conflation is a documented pattern in TenantMiddleware.cs:23 (tenantId = "default" fallback) + PowerFillRunHistoryService.cs:114 (TenantId = _tenant.TenantId on insert) + PowerFillRunHistoryService.cs:266 (Where(r => r.TenantId == _tenant.TenantId) on list).
Short-term disposition: Path γ (per PO 2026-04-19) — added Tenants__ps-demodata__ConnectionString env var to the staging API Deployment so the slot name matches the data tag (infra/azure/k8s/pssaas-staging/services.yaml). Staging now resolves X-Tenant-Id: ps-demodata correctly and surfaces the 12 historical rows. This unblocks the Phase 8 W2 PO milestone click-through but does NOT decouple the conflation — it just shifts the convention so the slot name and the data tag agree on 'ps-demodata' everywhere. The wart still exists; a future deploy or new tenant could re-trigger the same class of bug.
Long-term disposition (DEFERRED to Phase 9+ Architect-level work): decouple logical tenant identity from connection-string-slot routing. Two stable concepts, two storage locations:
- Tenant identity (e.g. UUID or stable customer-org slug like
watermark-tpo) — written to row columns; invariant across environments; tied to the customer-organization, never to the routing. - Routing map — separate config that maps tenant-identity → connection string, possibly including environment-specific overrides.
Concrete shape (proposed; subject to Architect refinement at Phase 9+):
TenantRegistryexposesTenant Resolve(string identity)returning(string TenantId, string ConnectionString)— both fields independent.- Config schema:
Tenants:<identity>:ConnectionStringANDTenants:<identity>:CanonicalId(the latter being the persisted column value, which can be the same across multiple connection-string aliases in different environments). - Migration path: existing
pfill_run_history.tenant_id='ps-demodata'rows get re-tagged to whatever the canonical customer identity becomes (e.g.'sandbox'or a UUID) once the convention is decided; a one-shot UPDATE script.
Banking pattern: PoC-stage multi-tenancy hygiene survives single-writer scenarios but breaks the moment a second route writes to the same DB. The class of bug is "convention conflation under low-corroboration count" — one writer means one convention, which feels like coherence but is actually under-tested. Generalization for the canonical: practice #13 Environment-Explicit Inventory should explicitly probe "does artifact X get written to with the same convention from every environment that can write it?" as a sub-cell. The Phase 8 W2 completion-report matrix did probe whether each tenant-routing CELL was verified, but did not probe convention-coherence ACROSS cells. Worth folding into the next process-discipline revision as a refinement.
Implication for Phase 9 (parallel validation against Desktop App): the harness will likely write into PS_DemoData from a third route (the harness itself, possibly running on AKS or a dev workstation). Whatever tenant_id convention it uses must agree with whatever the Phase 9 kickoff specifies — drift here would re-trigger A68 with the harness as the third writer.
Implication for production multi-tenancy: A68 must be resolved before the second customer organization is added (i.e. Phase 9+ when PS608 or any other live customer DB enters the picture). With one customer it doesn't bite; with two it's a guaranteed footgun.
2026-04-19 platform-tailwind note: A68 becomes architecturally cleaner under the upcoming Phase 8.5 work (PSSaaS joins ecosystem auth via platform-Keycloak). With Keycloak as the source of truth for authenticated user identity, the natural "tenant identity" anchor is the tenant_id claim on the OIDC token (which Keycloak owns and is invariant per customer-org), decoupled from whichever connection-string slot the request happens to route through. The long-term decoupling shape proposed above (separate Tenant Resolve(string identity) returning (TenantId, ConnectionString)) maps naturally onto "TenantId comes from Keycloak claim; ConnectionString comes from a per-tenant config map." Phase 8.5 design should treat A68 closure as a deliverable.
Needs confirmation from no one for the short-term Path γ disposition (PO-decided 2026-04-19, applied via kubectl + services.yaml edit). The long-term decoupling needs Architect-led design — most natural fold into Phase 8.5 (ecosystem auth integration); fallback target Phase 9+ kickoff carry-over.
A69: psp_powerfillUE Fires SqlException 207 'Invalid column name note_rate' On PS_DemoData When Invoked With Non-Empty Post-psp_powerfill_pool_guide State (NEW 2026-04-20, surfaced by Phase 9 first-run)
Claim: When psp_powerfillue is invoked via direct sqlcmd EXEC against PS_DemoData immediately after psp_powerfill_conset (writes 515 rows to pfill_powerfill_guide) and psp_powerfill_pool_guide (writes 515 rows to pfill_pool_guide) have populated their respective tables, UE fails within ~1s with SqlException 207: Invalid column name 'note_rate' (SQLSTATE 42S22). When psp_powerfillue is invoked in isolation against the post-PSSaaS-rebuild-empty state (pfill_powerfill_guide = 0 rows), it completes cleanly in ~2.7s. The failure path is state-dependent: UE has a code path that fires only when its inputs include the post-pool_guide 515-row state, NOT when invoked against the post-UE-rebuild empty state.
The PSSaaS-via-API run completed Successfully (status=Complete, ~30s, post_ue_*=0 per A66) on the SAME DB at the SAME time. Two possibilities worth Greg/Tom consultation:
- PSSaaS hits the SAME 207 internally and silently swallows it. The
RunStepResult.ErrorMessagefield for the UE step in the recordedpfill_run_history.response_jsonwould tell us; not yet captured in this first run. If true, this is itself a Phase 9 finding (PSSaaS reports Complete despite UE step Failing). - PSSaaS's UE invocation hits a DIFFERENT code path — perhaps because EF Core's
ExecuteSqlInterpolatedAsyncsets different SET options than pyodbc's default cursor SET options, and UE's behavior branches on those. Per A50 / F-6b-7, UE isWITH ENCRYPTION, which means SET options are captured at CREATE time and ignore session-level SET changes from the caller — so this hypothesis would point to UE's body itself behaving differently based on something other than SET.
Source: Phase 9 first end-to-end harness run (tools/parallel-validation/harness.py) against PS_DemoData on 2026-04-20. Empirical reproduction: 3 consecutive harness runs all surfaced the identical SqlException at the identical EXEC step. Diagnosis scripts at tools/parallel-validation/diagnose_a69.py (proc-body deployment + view-shape check) + diagnose_a69_v2.py (v_loan_loan_shipped + loan_shipped column verification — both have note_rate) + diagnose_a69_v3.py (UE-in-isolation success against post-rebuild-empty state). Full first-run report at docs-site/docs/devlog/2026-04-20-powerfill-phase-9-first-validation-run.md.
Confidence: High for the empirical observation (3-instance reproduction; clean isolation test). Open for the root cause (line-level UE archaeology not done this session).
Disposition: (b) Banked observation — Phase 9 first-run finding worth Greg/Tom consultation. The harness's verdict logic was updated post-first-run to honor this finding via a new RowVerdict.INCOMPARABLE value + a refined A66-aware rule that suppresses Match classification when EITHER side did not reach a comparable terminal state. The diff engine self-test gained Case 7 (asymmetric Failed) as a regression check. The first run's verdict report does NOT mis-claim parity — it surfaces A69 prominently in the TL;DR + a dedicated section.
Phase 9 framing implication: A69 is exactly the class of finding the Phase 9 harness was built to surface (per kickoff §"Reporting protocol": "If the harness reveals a real PSSaaS-vs-Desktop-App divergence beyond rounding tolerance — STOP and surface as A69+. The disposition is PO's call."). The harness earned its Phase 9 charter on its very first run. The Greg-demo narrative gains a load-bearing slide: "PSSaaS Phase 9 first run surfaced A69 — a state-dependent UE behavior asymmetry between the C# orchestration path and the bare-sqlcmd path. We've documented it for your input on root-cause and disposition."
Implication for Phase 9 follow-up sessions: the next Phase 9 dispatch (or a dedicated A69 investigation) should pin the failing UE query line via PRINT-instrumented copy of 011_*.sql + RAISERROR-on-each-step diagnostic + capture of pfill_run_history.response_json for the PSSaaS run that ostensibly Completed. Any of:
RunStepResult.ErrorMessagepopulated for theuestep → confirms hypothesis 1 (PSSaaS silently swallows)- PSSaaS's UE step succeeds without writing anything to ErrorMessage AND post_ue_* counters are 0 → confirms UE legitimately rebuilt-empty in PSSaaS's invocation but failed in sqlcmd's invocation, narrowing to hypothesis 2 (different code path)
- The line-level PRINT instrumentation reveals the failing query → root cause for either hypothesis
Implication for the customer-DB sweep: A69 may or may not fire on customer DBs. The harness's Capability × Environment matrix's "Customer DB" column carries a NOT MEASURED HERE cell for A69 reproducibility. Operator-driven post-Phase-9 runs against customer DBs without A54 triggers will both close the legacy-vs-fixed-body parity question AND empirically surface whether A69 is PS_DemoData-specific or generalizes.
Family relationship:
- A54 — A54 was the canonical first instance of the carve-out pattern; A69 is the canonical first instance of the harness pattern.
- A66 — A66 is the legitimate UE-rebuild-empty case; A69 is its asymmetric-failure cousin (one side rebuilds-empty per A66; the other fails to rebuild-anything because of A69).
- A50 / F-6b-7 — UE is
WITH ENCRYPTION; SET options captured at CREATE time. Limits the hypothesis space for "different code path" theories.
Needs confirmation from Greg/Tom (Phase 9 carry-over to either next Phase 9 follow-up session OR Greg-demo consultation). Disposition is PO's call.
A70: Frame D Hybrid Refines Phase 9's "Same Fixed Proc Body" Claim — PS_DemoData Has Mixed PSSaaS-Deployed + Legacy-Encrypted Procs (BANKED OBSERVATION 2026-04-20)
Claim: The Phase 9 kickoff and the early Phase 9 plan documents both treated PS_DemoData as having "the PSSaaS-fixed 009_*.sql proc body" deployed end-to-end. Empirical primary-source verification (per tools/parallel-validation/diagnose_a69.py query of sys.procedures on PS_DemoData 2026-04-20) reveals the actual state is mixed:
| Procedure | Encryption status on PS_DemoData | PSSaaS-deployed? |
|---|---|---|
psp_powerfill_pool_guide | plain | DEFINITELY YES — PSSaaS deploys without WITH ENCRYPTION per A50 + the A54 fix; the plain-text shape is the proof-of-deployment |
psp_powerfill_conset | WITH ENCRYPTION | Likely PSSaaS-deployed (PSSaaS deploys conset WITH ENCRYPTION per A50 + 008_*.sql) but indistinguishable from the legacy version since both are encrypted |
psp_powerfillue | WITH ENCRYPTION | Same as conset — likely PSSaaS-deployed via 011_*.sql per Phase 6d but indistinguishable from legacy |
psp_pfill_bx_cash_grids | WITH ENCRYPTION | Same — likely PSSaaS-deployed but indistinguishable |
psp_pfill_bx_settle_and_price | WITH ENCRYPTION | Same |
psp_pfill_ect_params | WITH ENCRYPTION | Same |
psp_pfill_trade_params | WITH ENCRYPTION | Same |
psp_pfill_insert4_pool_guide | WITH ENCRYPTION | Likely PSSaaS-deployed via 009_*.sql per A38 + the inner-EXEC dependency |
The kickoff's "same fixed proc body" claim survives in a refined form: Frame D's parity check verifies orchestration-equivalence against whatever proc bodies live on the target DB — which on PS_DemoData is a mixture of PSSaaS-deployed + (possibly) legacy-encrypted leftovers. The harness can't distinguish between "PSSaaS-deployed + encrypted" and "legacy + encrypted" via sys.objects. To resolve which is which would require either:
- Behavior diff — execute a known-distinguishing SQL on each proc and compare PSSaaS-side expected vs PS_DemoData-side actual (high cost; would need custom test cases per proc).
OBJECT_DEFINITIONquery — returns NULL forWITH ENCRYPTIONprocs, so this doesn't help.- Re-deploy + verify — explicit
DROP PROCEDURE + CREATE PROCEDUREfrom PSSaaS's008_*.sqletc., which would prove PSSaaS-side definition is what's running but might also be a behavior-change-by-omission if the legacy version had something PSSaaS doesn't.
Source: Empirical sys.procedures query on PS_DemoData via diagnose_a69.py (line "PSSaaS-deployed procs on PS_DemoData (encryption status):").
Confidence: High on the observation (sys.procedures query is canonical). Medium-Low on the "PSSaaS-deployed" disposition for the encrypted procs (per the table above; we believe yes per the deploy history but can't prove it from sys.objects alone).
Disposition: (b) Banked observation — refines Frame D's framing without invalidating it. The Phase 9 first-run report's Capability × Environment matrix already encodes the right level of honesty: "PSSaaS API path produces orchestration-equivalent allocations to direct sqlcmd path on PS_DemoData (Frame D Hybrid)" — this remains true regardless of which version of each proc is running, because both paths invoke whatever-is-there. The framing refinement matters for the Greg demo's level of claim: "PSSaaS's orchestration matches direct-sqlcmd's orchestration against the same set of proc bodies on PS_DemoData (whatever that set is)" is honest; "PSSaaS's port matches the legacy unmodified Desktop App proc body" is what would be a Capability Inflation overclaim AND is what the existing Capability × Environment matrix already explicitly marks NOT MEASURABLE HERE.
Implication for Phase 9 follow-up: the next session focused on A69 root-cause should ALSO take a 5-minute diversion to definitively resolve the "PSSaaS-deployed vs legacy" question for the WITH ENCRYPTION procs on PS_DemoData. Easiest method: behavior diff on a single distinguishing query per proc (e.g. psp_powerfill_pool_guide has the A54 fix; psp_powerfill_conset has the deploy-time PRINT statements per A32; etc.). Since pool_guide is plain (definitely PSSaaS), the ambiguity is on the OTHER 7 procs.
Implication for production cutover (Phase 10+): the customer-DB onboarding playbook MUST include an explicit "drop all legacy psp_powerfill_* + psp_pfill_* procs + redeploy from PSSaaS's *.sql files" step BEFORE the first PSSaaS run, OR document that legacy + PSSaaS-deployed-with-encryption are equivalent (which we can't prove without root-causing A70 first).
Family relationship:
- A50 / F-6b-7 — establishes that PSSaaS deploys conset / UE / etc. WITH ENCRYPTION, and the A54 fix proc (
009_*.sql) is the exception. A70 documents the consequence: PSSaaS-deployed-encrypted is indistinguishable from legacy-encrypted insys.objects. - A62 — PS_DemoData has a stale view definition (
pfillv_existng_pool_disposition14-col). A70 is the analog for procedure bodies: the deploy state of pre-PSSaaS-existing artifacts on PS_DemoData is ambiguous unless explicitly resolved.
Needs confirmation from no one to ship Phase 9 first run with Frame D Hybrid framing intact. Production cutover playbook (Phase 10+) needs explicit A70 disposition.