Architecting an AI Reconciliation Engine
A reconciliation engine that actually holds up in production is a pipeline, not a chat window. Once you've split the work between AI judgment and deterministic logic, the architecture follows directly from that split.
The flow
Data sources feed into normalization, which feeds into batch processing, which feeds into AI semantic matching, which feeds into deterministic validation, which produces an exception queue for a human to review. Six stages, each doing one job.
Phase 1: sanitize before you reconcile
Normalize incoming data before anything else touches it. Modern models can read a raw PDF or spreadsheet directly, but a dedicated preprocessing step, one that extracts text, strips irrelevant formatting, and converts everything into a clean CSV or JSON structure, produces meaningfully higher accuracy than skipping straight to the AI.
Phase 2: respect the context window
Feeding thousands of rows into a single AI call at once degrades both performance and accuracy. Batch the data instead, sized to stay comfortably within the model's context window, commonly around 50 rows depending on how wide the rows are. Every batch carries its own column headers, so the schema and the task instructions stay intact across the whole run rather than drifting as the conversation gets longer.
Phase 3: let AI match, not calculate
This is the part worth saying plainly, because it's the one people get backwards: the AI identifies candidate matches. It does not do the final math. The application takes the semantic matches the model proposes and combines them with deterministic business logic to calculate the actual variances.
Same principle as the decision framework, applied at the code level: judgment where judgment is needed, certainty everywhere else.
Phase 4: validate before anything ships
Before results reach a human, the application runs its own deterministic checks. Totals get recalculated independently. Required fields get verified. Output schemas get enforced. Anything inconsistent gets flagged rather than passed through.
The AI proposes. The software validates. Nothing reaches an analyst's desk that hasn't been checked twice, once by the model's judgment and once by code that doesn't have opinions.
The prompt itself has three jobs
The engine runs on a structured, three-part prompt rather than one long instruction. A system prompt sets the boundaries: what the engine is, what it's allowed to touch, what it will only output. A task prompt defines the specific comparison being made. A format prompt forces the output into a fixed, parseable structure, explicitly ruling out conversational text.
Separating these three jobs is what keeps the engine predictable across thousands of runs instead of subtly drifting the way a single freeform prompt would.
None of this matters if the system can't explain itself when something looks wrong. What the engine does with ambiguity, partial matches, and low-confidence calls is what actually earns enterprise trust, and it's the next piece in this series.
Adapted from the full white paper on designing AI reconciliation systems. Download the PDF.
Rosemarie Withee has spent thirteen years helping operations teams get real work out of their software, first Microsoft 365, now AI. She's written six books for Wiley and builds AI products at Portal Integrators.