When will SWE-bench be solved? | Manifold

When will SWE-bench be solved?

11

Ṁ100Ṁ544

2030

0.9%

2024

0.9%

2025

76%

2026

7%

2027

7%

2028

7%

2029

2%

later

Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.

Market context

Technical AI Timelines

Get

1,000

to start trading!

Sort by:

bought Ṁ5 YES

Relatively significant chance of never (which resolves at later). Frontier LLMs are just going to get more and more contaminated with this benchmark's test data - at some point authors might just stop tracking

People are also trading

What will be the highest score on the SWE-bench pro private set before 2027?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

When Will 9x9 Go Be Solved?

Best SWE-Bench Pro public score by June 30, 2026

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

Top SWE-Bench Pro score by Jan 1, 2027?

Will Claude Sonnet 5 exceed 85% on SWE-bench verified?

In what year will go be solved?

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

Related questions

What will be the highest score on the SWE-bench pro private set before 2027?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

When Will 9x9 Go Be Solved?

Best SWE-Bench Pro public score by June 30, 2026

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

Top SWE-Bench Pro score by Jan 1, 2027?

Will Claude Sonnet 5 exceed 85% on SWE-bench verified?

In what year will go be solved?

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

© Manifold Markets, Inc.•Terms•Privacy