ADR-022: PowerFill Allocation Algorithm

Status

Proposed

Context

The Desktop App PowerFill engine solves a constrained optimization problem: given a set of loans and a set of trades with eligibility constraints and tolerance bands, assign loans to trades/pools to maximize fulfillment and delivery value.

The legacy implementation in psp_powerfill_conset uses an iterative multi-pass allocation approach:

1. Walk constraints in priority order
2. For each constraint, build a candidate set of loan-trade pairs scored by price (or price + carry)
3. Multiple allocation passes: exact fit → best fit → fill remaining → orphan handling
4. Loans not allocated after the final pass become "kickouts"

This is approximately 19K lines of T-SQL that has been tuned against real customer pipelines for 15+ years. The tuning is largely invisible — we don't know which lines of code embody critical domain knowledge versus accidental complexity.

Porting requires choosing between three algorithm approaches.

Option A: Port the Iterative Passes Verbatim

Translate the existing T-SQL allocation passes into PSSaaS stored procedures (per ADR-021) with minimal semantic change. Behavior matches Desktop App 1:1. Performance should be comparable since it's the same algorithm on similar infrastructure.

Option B: Rewrite as Linear Programming

Formulate PowerFill as a mixed-integer linear program (MILP) and solve with a commercial or open-source solver (Gurobi, CPLEX, Google OR-Tools, CBC):

• Variables: one binary per (loan, trade) pair
• Constraints: trade target ± tolerance, loan allocated once, securitization rules, priority ordering
• Objective: maximize total delivery value

Theoretical benefits: provably optimal solutions, clean mathematical formulation, decades of solver tooling.

Option C: Constraint Satisfaction Programming (CSP)

Model as a constraint satisfaction problem with a CSP solver (OR-Tools CP-SAT, Choco, MiniZinc). Supports richer non-linear constraints than LP.

The Core Question

Is PowerFill's legacy algorithm principled (we could replace it with LP/CSP and get equivalent or better results) or tuned (the passes embody specific domain choices that solvers wouldn't replicate)?

Evidence from the legacy code:

• Multiple distinct passes (exact fit, best fit, fill remaining, orphans) — suggests deliberate prioritization not easily captured in a single objective function
• Priority-ordered constraint iteration — higher-priority constraints consume capacity before lower-priority ones get a chance
• Carry cost scoring that adjusts but doesn't eliminate — suggests a weighted objective rather than a hard constraint
• Kickout handling with semantic reasons — suggests a user expectation of "here's why this loan didn't allocate" rather than a binary solution/no-solution

This pattern is characteristic of a greedy algorithm with domain-specific passes, not a principled optimization. Tuning the priority order and pass semantics IS the business logic.

Decision

Adopt Option A: Port the iterative passes verbatim.

Specifically

• Translate psp_powerfill_conset and psp_powerfillUE from Desktop App T-SQL to PSSaaS T-SQL with minimal semantic changes
• Preserve the multi-pass structure: exact fit → best fit → fill remaining → orphan handling
• Preserve the constraint priority iteration order
• Preserve the carry cost scoring approach (price or price+carry, configurable)
• Preserve the kickout semantics (loans that fail allocation are captured with reasons)
• Document every interpretation made during the port in the Assumptions Log
• Validate the port against Desktop App output on PS_DemoData before declaring parity

Explicit Non-Goals for Phase 0-9

• Not rewriting as MILP
• Not introducing a constraint solver library
• Not "improving" the algorithm based on our interpretation
• Not collapsing the multiple passes into a single optimization

LP/CSP as a Phase 2 Modernization Candidate

A future ADR may supersede this one by migrating to LP or CSP. The prerequisites for that decision are:

1. PSSaaS PowerFill is in production with at least one customer
2. Tom or Greg has reviewed the port and confirmed correctness
3. A pilot LP/CSP implementation produces equivalent or better results on customer pipelines
4. The business case (improved margin, faster runs, fewer kickouts) justifies the rewrite cost

Until then, the iterative passes are the algorithm.

Consequences

Positive

• Lowest risk of behavioral drift from the Desktop App
• Customer parallel validation is meaningful — we can show identical output on identical inputs, building trust
• Tuning knowledge preserved — whatever business logic lives in the passes stays intact, even if we don't fully understand it
• Fastest path to parity — no algorithm rewrite means fewer unknowns
• Tom/Greg review is tractable — they can compare our T-SQL to theirs line-by-line if needed

Negative

• No theoretical guarantees — greedy algorithms don't guarantee optimality; LP would
• Carries forward any legacy bugs — 15 years of tuning may include tuning around bugs we'll dutifully reproduce
• Harder to "prove" correctness — we can only validate empirically, not mathematically
• Bounded improvement potential — the algorithm's ceiling is wherever the Desktop App is today; LP might have a higher ceiling

Risks and Mitigations

Risk	Mitigation
Ported algorithm produces different results than Desktop App	Parallel validation on PS_DemoData; document any deltas and classify as bug, enhancement, or acceptable variation
Legacy algorithm has known bugs	Assumptions log captures any suspicious behavior; Tom/Greg review confirms intended vs. buggy behavior
Performance regression on larger pipelines	Benchmark suite against PS_DemoData and synthetic large-scale data; optimize specific procs if needed
Inability to improve algorithm in response to customer feedback	Phase 2 modernization path documented above; can supersede this ADR when warranted

Alternatives Considered and Rejected

Option B (MILP Rewrite)

Rejected because: the multi-pass structure and priority-ordered constraint iteration aren't naturally expressed as LP. Converting them requires making semantic choices ("what does exact fit even mean in a single objective function?") that would diverge from Desktop App behavior in ways we can't predict. The risk is high and the reward is theoretical.

Separately, MILP solvers have licensing cost considerations (Gurobi, CPLEX) or performance tradeoffs (CBC, GLPK) that add operational complexity without a proven business benefit.

Option C (CSP)

Rejected for similar reasons as Option B, plus: CSP tooling (OR-Tools CP-SAT) is excellent for scheduling and resource allocation but doesn't naturally match the financial scoring logic (price + carry) that PowerFill uses. We'd end up either approximating the scoring or bolting scoring onto a CSP solver, which loses CSP's native benefits.

Option D (Custom Greedy Rewrite in C#)

Not seriously considered. If we're keeping the greedy multi-pass algorithm, T-SQL is the right language (per ADR-021). Re-implementing the greedy algorithm in C# would be the worst of both worlds — same algorithm, different language, different performance characteristics.

Relationship to Other ADRs

• ADR-021 (Port Strategy) — this ADR confirms the core allocation engine stays in T-SQL stored procedures per the hybrid strategy
• ADR-023 (Constraint Model) — this ADR picks the algorithm; ADR-023 picks the constraint data representation
• ADR-004 (Modular Monolith) — the PowerFill module is internally coherent; its algorithm choice is encapsulated

Revision Triggers

This ADR should be revisited if:

• Tom/Greg critique reveals the iterative passes have fundamental flaws
• Customer feedback indicates dissatisfaction with allocation quality that LP might solve
• Performance measurements at scale show the greedy algorithm is a bottleneck
• A compelling LP/CSP prototype produces materially better results on real customer pipelines
• An existing financial-domain solver library emerges that captures the PowerFill pattern natively