The frontier model you choose today is unlikely to be the best one a year from now. For buy-side credit teams, that creates a structural risk: if your workflow is built around one model, every model transition carries a switching cost.
I ran the same ABF credit memo task across three models this week: identical prompt, identical 14-document dataroom, blind scoring against a ground-truth data dictionary. The spread was 13 points. The top model caught a borrowing-base shortfall, a pre-dated compliance certificate, and a blank officer signature that the others missed.
That accuracy gap matters in the short term. A 13-point spread across models on a live deal is not noise. But that was not the main takeaway from the exercise.
The benchmark was easy to run because the extraction pipeline, prompt structure, and scoring rubric were model-agnostic. Same inputs, different engine. The architecture made the comparison possible in a single afternoon.
That is the real capability: not choosing the right model once, but building infrastructure where the model is a variable, not a dependency. When the next frontier release outperforms today's best on credit document extraction, a model-agnostic system can capture that improvement with minimal friction. A workflow built around one model's quirks has to be partially rebuilt.
The pattern in financial services has been the opposite. Teams find a model that works, tune prompts around its specific behavior, and build workflows that are implicitly dependent on that model's output format and failure modes. That works until the model is deprecated or a better alternative emerges. Then the switching cost is real.
The model layer is becoming the most commoditized part of the stack. Frontier capability is increasingly available at comparable cost across providers, and the gap between models narrows faster than firms can redesign workflows to exploit it. The durable investment is the layer underneath: extraction pipelines, context structures, evaluation harnesses, and scoring rubrics that work regardless of which engine is running.
Build for model portability. The benchmark run this week took three hours. That is what model-agnostic infrastructure looks like in practice.