FinanceBench
Financial questions answered using public-company filings and supporting evidence.
FinanceBench · November 2023 · published research
Evaluation setup
Human-reviewed results with context-first prompting. Compare complete model-plus-retrieval configurations, not model names alone. Oracle receives the evidence pages and is a reference condition, not an equivalent retrieval system.
Reading this comparison
Higher values indicate better performance under the named study's scoring rules. Published model names are retained. These are historical snapshots, not tests of later versions with similar names.
Boundaries
A benchmark answer score does not establish a model's readiness to deliver a financial diligence engagement. It does not measure CPA sign-off, evidence retention, access controls, or performance on your transaction documents.
Provenance
Transcribed from the linked primary source on September 12, 2026. Evaluation was performed by the source authors, not Finsider. The downloadable CSV includes each value, configuration, and source URL.
Performance Comparison
Correct answers (%) · source rounding
GPT-4-Turbo · Oracle
85
GPT-4-Turbo · Long Context
79
Claude2 · Long Context
76
GPT-4-Turbo · Single Vector Store
50
Llama2 · Single Vector Store
41
GPT-4-Turbo · Shared Vector Store
19
Llama2 · Shared Vector Store
19
Seven open-book configurations from Table 2, with 150 questions per configuration. Closed-book results are omitted.
Rank uses displayed scores, and ties share rank. Bars use a 0–100 scale. The source table provides no confidence intervals, so none are drawn.
Source: FinanceBench · November 2023