The Most Expensive AI Model Isn’t Always the Best

We put 11 leading AI models through 15 categories of real-world investment-agent tasks. The results looked very different from general-purpose benchmarks—and showed why the most expensive model is not necessarily the best model for the job.

Consider a common portfolio-risk request. A user provides one month of fund NAV data and only a few months of monthly closing prices for the portfolio’s U.S. equity holdings. One model calculates a 63.3% annualized return and a Sharpe ratio of 6.0, then adds a disclaimer. Another correctly identifies that the data is insufficient—but refuses to do any useful work at all.

The better response lies between those extremes: calculate what the available data can support, explain what cannot be determined, and give the user a useful partial result without pretending the missing information exists.

DrivenBench was built to evaluate exactly these product-level behaviors. This article explains what we tested, how we scored the models, what the results reveal, and which models fit different investment workflows. Driven’s Auto mode uses the same evidence to choose a model by scenario.

What We Test

Most general-purpose benchmarks are built around puzzles with a clear right or wrong answer. Investment agents face a different kind of challenge. They must interpret financial conventions, work with incomplete data, use tools reliably, preserve account safety, and know when to stop.

A model may need to distinguish the Hong Kong market convention of 亿 from the U.S. convention of billion; report a quote API failure honestly; create a one-off task on an earnings date; or follow an account confirmation workflow exactly.

These tasks rarely have a single ideal sentence as the answer. Retrieving the right information, stating data gaps, and delivering the useful portion of a task can all be correct. Pretending that missing data exists is not.

Each evaluation therefore defines required behaviors and prohibited behaviors. Results are assessed with a fixed judge model alongside deterministic checks where possible.

How We Test

The benchmark covers 15 capability areas, including numerical reasoning, filings and market data, external research, data integrity, scheduled tasks, trading and accounts, file delivery, and delegation. Many scenarios are grounded in real production incidents and user workflows.

Capability evaluations are run three times and contribute to the ranking. Separate regression checks cover critical failure modes; they are reported independently and do not affect rank. Runs are scheduled in advance, incomplete work counts as a failure, and failed runs cannot be replaced. Only genuine environment failures—such as a network interruption—are invalidated and rerun.

Cost is calculated as an upper bound using published prices without cache discounts. Latency is the observed runtime.

Results

• 1 (tie). Claude Sonnet 5 — 93.9% capability score, $58 benchmark cost, 30s median latency, high reasoning. The most consistent model in the benchmark: no task failed in all three runs.
• 1 (tie). Kimi K3 — 93.9%, $59, 59s, max reasoning. It matched Sonnet’s score but took roughly twice as long.
• 3. Claude Opus 5 — 90.9%, $160, 40s, high reasoning. The most expensive model produced clean execution, but lost points for not following exact instructions.
• 4. DeepSeek V4 Pro (0813) — 89.7%, $31, 24s, high reasoning. It delivered 96% of the top score at roughly half the cost.
• 5. Grok 4.6 — 89.1%, $74, 75s, high reasoning. A solid score, but the slowest model in the benchmark and not inexpensive.
• 6. GLM 5.2 — 87.3%, $26, 23s, high reasoning. A capable mid-priced option, though it sometimes wasted time repeatedly retrying invalid commands.
• 7. GPT-5.6 Sol — 85.5%, $93, 27s, low reasoning. Honest but often too cautious: when uncertain, it sometimes refused to act and returned little useful work.
• 8. Gemini 3.7 Flash — 81.8%, $14, 23s, medium reasoning. A low-cost mid-tier model that occasionally reported unfinished steps as completed.
• 9. GPT-5.6 Luna — 79.4%, $3.6, 19s, low reasoning. Extremely inexpensive and more likely to avoid action than make a mistake; best suited to simple tasks.
• 10. DeepSeek V4 Flash (0731) — 77.6%, $3.0, 20s, high reasoning. It readily accepted tasks, but complex workflows often stopped halfway.
• 11. GPT-5.6 Terra — 76.4%, $34, 16s, low reasoning. The fastest model, but the least reliable—and the only one that damaged an existing setting by clearing a scheduled task’s name while pausing it.

Notes on the numbers

1. The capability score is the equally weighted average pass rate across capability evaluations.
2. Cost is the published-price upper bound for three runs of every capability evaluation, excluding cache discounts; observed billed cost was about one-third of this upper bound because of cache hits.
3. Latency is the median runtime per evaluation.
4. DeepSeek costs use pricing from before August 16, 2026. At current peak pricing, V4 Flash would cost about $10 in total; V4 Pro is largely unchanged.
5. Reasoning effort reflects the settings used in Driven at evaluation time and is not directly comparable across providers. GPT models used low, following OpenAI’s recommendation for latency-sensitive, tool-using workflows. Gemini 3.7 Flash used medium, its family default; minimal is not supported.

Behind the Scores: Three Details Worth Knowing

Two Models Tied for First—but They Feel Very Different

Claude Sonnet 5 and Kimi K3 finished with the same score, but their operating profiles were not the same. Sonnet’s median response time was about 30 seconds. Kimi’s was about a minute, with its slowest tasks approaching three minutes.

Sonnet’s failures were intermittent: no task failed on all three runs. In two cases, it completed the work but ran out of time before writing the final answer. Kimi’s more concerning miss was one of judgment: after querying an empty data source, it still designed a monitoring plan as if the data existed.

Two Cheap Models, Opposite Failure Modes

DeepSeek V4 Flash tends to charge ahead. It will attempt almost anything, but complex tasks can stop halfway, especially when it fails to use actual column names or learn from tool errors.

GPT-5.6 Luna tends to hold back. It sometimes invents extra confirmation requirements and then does nothing. In the sparse-data portfolio example, V4 Flash produced an implausible return and added a disclaimer; Luna refused even the parts that could have been calculated safely.

Both can be useful for one-turn quotes and summaries. Use more caution for files, account operations, and multi-step workflows.

The Most Expensive Model Lost in an Unexpected Place

Claude Opus 5 had a clean process. It did not brute-force tools or abandon workflows halfway through. Its losses came mostly from instruction compliance.

In one case, it recognized a date cutoff but still used data from after the cutoff. In another, it “verified” a save by reading back from the same location it had just written to—a self-echo rather than independent confirmation.

The $160 benchmark cost bought a clean process, but not the strongest instruction adherence. Sonnet cost roughly one-third as much and scored higher.

Which Model Should You Choose?

• Chat and interactive research: Claude Sonnet 5. Top-tier capability, about 30 seconds median latency, and no consistently weak task category.
• Default or cost-sensitive workflows: DeepSeek V4 Pro (0813). About four points behind the leaders, roughly half the cost, and faster.
• Background batch work, scheduled tasks, and deep research: Kimi K3. Tied for first; its slower responses are less costly when the work runs in the background.
• High-volume, lightweight tasks: GPT-5.6 Luna. Roughly one-sixteenth of the leaders’ cost and suitable for simple work where caution is acceptable.
• Not recommended as a default: GPT-5.6 Terra. Its confirmed failure could damage existing settings, and its cost is not especially low.

Expensive models are not automatically the wrong choice. Opus can still be valuable where a clean process matters more than cost. Grok and Sol are also capable models, but this benchmark found stronger alternatives at similar price points for investment-agent work.

Limitations

• The benchmark evaluates individual tasks lasting minutes, not multi-day monitoring workflows.
• Cost uses the maximum published price. Actual billed cost can be lower because of caching, and provider discounts vary.
• Three runs per capability task are enough to identify stable differences, but not to estimate the probability of rare failures precisely.
• The suite grows as new real-world incidents emerge, and rankings will change as model versions evolve.

If You’d Rather Not Choose

Driven’s Auto mode selects a model by scenario using the same benchmark evidence. Interactive chat prioritizes speed; background work prioritizes reliability. As the benchmark expands, Auto mode can continue to update its choices so users do not have to chase every leaderboard change.

Results reflect the tested model versions and conditions, may change over time, and are provided for comparative research—not investment advice.

View the full benchmark results: https://driven.ai/drivenbench