Financial AI will be judged by the output, not the model

By Charles Poliacof, Chief Commercial Officer

The capabilities of foundation models matter enormously. Better models make more ambitious financial work possible. But two products using the same model can differ substantially in the quality of their output, the intervention they require and the cost of completing an assignment. 

The difference lies in the harness and surrounding systems: how they connect models to financial data, tools and institutional context, execute the work and check the result. Together, those systems determine whether an output can be used in a valuation, a pitch or a client meeting. 

Model selection answers only part of the question. Five questions help assess the financial product built around it. 

Can it understand the job? 

“Update the trading comps” sounds like a straightforward instruction. In practice, it leaves much unstated. Which peers belong in the set? Which consensus period and source should be used? Does the team use reported or adjusted EBITDA? How does it treat leases and minority interests? 

The system needs to apply the team’s established approach, make material assumptions visible and flag important unresolved choices. The analyst should be able to inspect and change those choices without rebuilding the work. 

Context matters just as much. “Prepare me for tomorrow’s meeting with Nike” should produce different work at two banks because their relationships, previous pitches and current opportunities differ. Within the same firm, a banker pursuing a mandate needs different preparation from a research analyst discussing an investment view. 

That context should shape what the system retrieves, analyzes and produces. 

Can it retrieve and reconcile the information? 

Connecting to FactSet, S&P Capital IQ or PitchBook does not establish equivalent access across products. Coverage and entitlements matter, alongside the ability to interpret the information correctly. 

“Next year’s earnings” illustrates the challenge. The answer depends on fiscal versus calendar periods, how earnings are defined and when consensus was captured. A number can be correct in its source and still be wrong for the analysis. 

A financial semantic layer helps make these definitions and relationships explicit, giving the system a consistent basis for interpreting the request. 

The broader system must preserve that meaning through the analysis. That includes carrying the source, period, currency and definition of each material figure through the work. When a filing, a market-data feed and internal research disagree, it should investigate whether the difference reflects timing, methodology or an inconsistency, and make unresolved differences visible. 

Can it plan and execute? 

Good orchestration combines the firm’s methods with the right tool for each step. 

A research team’s first take after earnings might retrieve consensus through an API, calculate variances using established formulas and use a model to interpret management commentary. The system then assembles the analysis according to the team’s framework and routes the draft for review. 

Retrieving a figure or calculating a multiple can run without an LLM call, avoiding unnecessary inference costs. Defined software operations provide reproducible results; models can handle the reasoning and interpretation the assignment requires. 

The team’s approach to guidance, peer implications and approvals should be reusable across assignments. The product should preserve that workflow while selecting and testing suitable models across providers. 

In a recent banking session, analysts estimated that without vertical AI, recreating a linked slide could take about an hour. They wanted a versioned, editable Excel backup alongside the presentation. Delivery needs to preserve the calculations and source links, alongside consistency between the workbook and slides. 

Can it validate and document? 

For a trading-comps update, validation means checking the agreed peers, forecast periods and metric definitions, reconciling the multiples to the underlying calculations and preserving the workbook’s formulas. Citations beside the inputs establish only part of that. 

These requirements should become repeatable checks on completed work. Some can run through conventional software; others require model-assisted assessment or professional review. Changes to models, prompts or workflows should be tested against the same requirements. 

The record should connect source data to adjustments, calculations and the final output, identifying failed checks and unresolved exceptions. That gives the reviewer a basis for accepting or changing the work. 

Can it act proactively? 

Suppose a company announces plans to sell a division. A system monitoring corporate announcements could identify clients whose acquisition criteria align with the asset, drawing on their stated priorities, previous transactions and the bank’s relationship history. 

It could prepare a briefing on strategic fit, potential obstacles and relevant transaction precedents, alongside a suggested angle for approaching the client. The banker reviews the idea and decides whether and how to pursue it. 

This requires the system to connect an external event to knowledge the bank already holds, recognize which clients may be affected and initiate the relevant analysis while there is still time to act. The product determines when the model’s capabilities should be deployed and what work to set in motion. 

Conclusion 

A vertical provider’s contribution is to engineer financial expertise into repeatable execution: defined analytical methods, financial calculations and checks that hold up across real assignments. Those capabilities should exist before a bank adds its own conventions and institutional context. Better models can then extend the range and complexity of work the application undertakes, building on an established financial foundation. Banks should judge the depth of that specialization by what is implemented and demonstrated in completed work.