Finsider Labs
All leaderboards

FinanceBench

Financial questions answered using public-company filings and supporting evidence.

FinanceBench · November 2023 · published research

Evaluation setup

Human-reviewed results with context-first prompting. Compare complete model-plus-retrieval configurations, not model names alone. Oracle receives the evidence pages and is a reference condition, not an equivalent retrieval system.

Reading this comparison

Higher values indicate better performance under the named study's scoring rules. Published model names are retained. These are historical snapshots, not tests of later versions with similar names.

Boundaries

A benchmark answer score does not establish a model's readiness to deliver a financial diligence engagement. It does not measure CPA sign-off, evidence retention, access controls, or performance on your transaction documents.

Provenance

Transcribed from the linked primary source on September 12, 2026. Evaluation was performed by the source authors, not Finsider. The downloadable CSV includes each value, configuration, and source URL.

Performance Comparison

Correct answers (%) · source rounding

1

GPT-4-Turbo · Oracle

85

2

GPT-4-Turbo · Long Context

79

3

Claude2 · Long Context

76

4

GPT-4-Turbo · Single Vector Store

50

5

Llama2 · Single Vector Store

41

6

GPT-4-Turbo · Shared Vector Store

19

6

Llama2 · Shared Vector Store

19

050100

Seven open-book configurations from Table 2, with 150 questions per configuration. Closed-book results are omitted.

Rank uses displayed scores, and ties share rank. Bars use a 0–100 scale. The source table provides no confidence intervals, so none are drawn.

Source: FinanceBench · November 2023